Feb 17 18:07:08 [5320] node-3.test.domain.local corosync notice [MAIN ] main Corosync Cluster Engine ('2.3.4'): started and ready to provide service. Feb 17 18:07:08 [5320] node-3.test.domain.local corosync info [MAIN ] main Corosync built-in features: dbus testagents rdma watchdog augeas pie relro bindnow Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize Token Timeout (3650 ms) retransmit timeout (357 ms) Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize token hold (275 ms) retransmits before loss (10 retrans) Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize join (60 ms) send_join (0 ms) consensus (3600 ms) merge (200 ms) Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize downcheck (1000 ms) fail to recv const (2500 msgs) Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize seqno unchanged const (30 rotations) Maximum network MTU 1401 Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize window size per rotation (50 messages) maximum messages per rotation (20 messages) Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize missed count const (5 messages) Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize send threads (2 threads) Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token expired timeout (357 ms) Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token problem counter (2000 ms) Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP threshold (10 problem count) Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP multicast threshold (100 problem count) Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP automatic recovery check timeout (1000 ms) Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP mode set to none. Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize heartbeat_failures_allowed (0) Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize max_network_delay (50 ms) Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0 Feb 17 18:07:08 [5320] node-3.test.domain.local corosync notice [TOTEM ] totemnet_instance_initialize Initializing transport (UDP/IP Unicast). Feb 17 18:07:08 [5320] node-3.test.domain.local corosync notice [TOTEM ] init_nss Initializing transmit/receive security (NSS) crypto: none hash: none Feb 17 18:07:08 [5320] node-3.test.domain.local corosync notice [TOTEM ] timer_function_netif_check_timeout The network interface is down. Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [TOTEM ] main_iface_change_fn Created or loaded sequence id 0.127.0.0.1 for this ring. Feb 17 18:07:08 [5320] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration map access [0] Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cmap [0] Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:07:08 [5320] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cmap Feb 17 18:07:08 [5320] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration service [1] Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cfg [1] Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:07:08 [5320] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cfg Feb 17 18:07:08 [5320] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync cluster closed process group service v1.01 [2] Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cpg [2] Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:07:08 [5320] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cpg Feb 17 18:07:08 [5320] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync profile loading service [4] Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on pload [4] Feb 17 18:07:08 [5320] node-3.test.domain.local corosync warning [WD ] setup_watchdog No Watchdog, try modprobe Feb 17 18:07:08 [5320] node-3.test.domain.local corosync info [WD ] wd_scan_resources no resources configured. Feb 17 18:07:08 [5320] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync watchdog service [7] Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on wd [7] Feb 17 18:07:08 [5320] node-3.test.domain.local corosync notice [QUORUM] quorum_exec_init_fn Using quorum provider corosync_votequorum Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_readconfig Reading configuration (runtime: 0) Feb 17 18:07:08 [5320] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_read_nodelist_configuration No nodelist defined or our node is not in the nodelist Feb 17 18:07:08 [5320] node-3.test.domain.local corosync crit [QUORUM] quorum_exec_init_fn Quorum provider: corosync_votequorum failed to initialize. Feb 17 18:07:08 [5320] node-3.test.domain.local corosync error [SERV ] corosync_service_defaults_link_and_init Service engine 'corosync_quorum' failed to load for reason 'configuration error: nodelist or quorum.expected_votes must be configured!' Feb 17 18:07:08 [5320] node-3.test.domain.local corosync error [MAIN ] _corosync_exit_error Corosync Cluster Engine exiting with status 20 at service.c:356. Feb 17 18:17:40 [4598] node-3.test.domain.local corosync notice [MAIN ] main Corosync Cluster Engine ('2.3.4'): started and ready to provide service. Feb 17 18:17:40 [4598] node-3.test.domain.local corosync info [MAIN ] main Corosync built-in features: dbus testagents rdma watchdog augeas pie relro bindnow Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize Token Timeout (3650 ms) retransmit timeout (357 ms) Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize token hold (275 ms) retransmits before loss (10 retrans) Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize join (60 ms) send_join (0 ms) consensus (3600 ms) merge (200 ms) Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize downcheck (1000 ms) fail to recv const (2500 msgs) Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize seqno unchanged const (30 rotations) Maximum network MTU 1401 Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize window size per rotation (50 messages) maximum messages per rotation (20 messages) Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize missed count const (5 messages) Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize send threads (2 threads) Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token expired timeout (357 ms) Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token problem counter (2000 ms) Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP threshold (10 problem count) Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP multicast threshold (100 problem count) Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP automatic recovery check timeout (1000 ms) Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP mode set to none. Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize heartbeat_failures_allowed (0) Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize max_network_delay (50 ms) Feb 17 18:17:40 [4598] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0 Feb 17 18:17:40 [4598] node-3.test.domain.local corosync notice [TOTEM ] totemnet_instance_initialize Initializing transport (UDP/IP Unicast). Feb 17 18:17:40 [4598] node-3.test.domain.local corosync notice [TOTEM ] init_nss Initializing transmit/receive security (NSS) crypto: none hash: none Feb 17 18:17:41 [4598] node-3.test.domain.local corosync notice [TOTEM ] timer_function_netif_check_timeout The network interface is down. Feb 17 18:17:41 [4598] node-3.test.domain.local corosync debug [TOTEM ] main_iface_change_fn Created or loaded sequence id 0.127.0.0.1 for this ring. Feb 17 18:17:41 [4598] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration map access [0] Feb 17 18:17:41 [4598] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cmap [0] Feb 17 18:17:41 [4598] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:17:41 [4598] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cmap Feb 17 18:17:41 [4598] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration service [1] Feb 17 18:17:41 [4598] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cfg [1] Feb 17 18:17:41 [4598] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:17:41 [4598] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cfg Feb 17 18:17:41 [4598] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync cluster closed process group service v1.01 [2] Feb 17 18:17:41 [4598] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cpg [2] Feb 17 18:17:41 [4598] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:17:41 [4598] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cpg Feb 17 18:17:41 [4598] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync profile loading service [4] Feb 17 18:17:41 [4598] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on pload [4] Feb 17 18:17:41 [4598] node-3.test.domain.local corosync warning [WD ] setup_watchdog No Watchdog, try modprobe Feb 17 18:17:41 [4598] node-3.test.domain.local corosync info [WD ] wd_scan_resources no resources configured. Feb 17 18:17:41 [4598] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync watchdog service [7] Feb 17 18:17:41 [4598] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on wd [7] Feb 17 18:17:41 [4598] node-3.test.domain.local corosync notice [QUORUM] quorum_exec_init_fn Using quorum provider corosync_votequorum Feb 17 18:17:41 [4598] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_readconfig Reading configuration (runtime: 0) Feb 17 18:17:41 [4598] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_read_nodelist_configuration No nodelist defined or our node is not in the nodelist Feb 17 18:17:41 [4598] node-3.test.domain.local corosync crit [QUORUM] quorum_exec_init_fn Quorum provider: corosync_votequorum failed to initialize. Feb 17 18:17:41 [4598] node-3.test.domain.local corosync error [SERV ] corosync_service_defaults_link_and_init Service engine 'corosync_quorum' failed to load for reason 'configuration error: nodelist or quorum.expected_votes must be configured!' Feb 17 18:17:41 [4598] node-3.test.domain.local corosync error [MAIN ] _corosync_exit_error Corosync Cluster Engine exiting with status 20 at service.c:356. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [MAIN ] main Corosync Cluster Engine ('2.3.4'): started and ready to provide service. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync info [MAIN ] main Corosync built-in features: dbus testagents rdma watchdog augeas pie relro bindnow Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize Token Timeout (3650 ms) retransmit timeout (357 ms) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize token hold (275 ms) retransmits before loss (10 retrans) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize join (60 ms) send_join (0 ms) consensus (3600 ms) merge (200 ms) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize downcheck (1000 ms) fail to recv const (2500 msgs) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize seqno unchanged const (30 rotations) Maximum network MTU 1401 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize window size per rotation (50 messages) maximum messages per rotation (20 messages) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize missed count const (5 messages) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize send threads (2 threads) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token expired timeout (357 ms) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token problem counter (2000 ms) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP threshold (10 problem count) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP multicast threshold (100 problem count) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP automatic recovery check timeout (1000 ms) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP mode set to none. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize heartbeat_failures_allowed (0) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize max_network_delay (50 ms) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [TOTEM ] totemnet_instance_initialize Initializing transport (UDP/IP Unicast). Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [TOTEM ] init_nss Initializing transmit/receive security (NSS) crypto: none hash: none Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [TOTEM ] timer_function_netif_check_timeout The network interface [192.168.0.4] is now up. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] main_iface_change_fn Created or loaded sequence id 18.192.168.0.4 for this ring. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration map access [0] Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cmap [0] Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:26:07 [5459] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cmap Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration service [1] Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cfg [1] Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:26:07 [5459] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cfg Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync cluster closed process group service v1.01 [2] Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cpg [2] Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:26:07 [5459] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cpg Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync profile loading service [4] Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on pload [4] Feb 17 18:26:07 [5459] node-3.test.domain.local corosync warning [WD ] setup_watchdog No Watchdog, try modprobe Feb 17 18:26:07 [5459] node-3.test.domain.local corosync info [WD ] wd_scan_resources no resources configured. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync watchdog service [7] Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on wd [7] Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [QUORUM] quorum_exec_init_fn Using quorum provider corosync_votequorum Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_readconfig Reading configuration (runtime: 0) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_readconfig ev_tracking=0, ev_tracking_barrier = 0: expected_votes = 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=1, expected_votes=3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: Yes Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync vote quorum service v1.0 [5] Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on votequorum [5] Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:26:07 [5459] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: votequorum Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync cluster quorum service v0.1 [3] Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on quorum [3] Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:26:07 [5459] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: quorum Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [TOTEM ] totemudpu_member_add adding new UDPU member {192.168.0.6} Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [TOTEM ] totemudpu_member_add adding new UDPU member {192.168.0.4} Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [TOTEM ] totemudpu_member_add adding new UDPU member {192.168.0.5} Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_gather_enter entering GATHER state from 15(interface change). Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_commit_token_create Creating commit token because I am the rep. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] old_ring_state_save Saving state aru 0 high seq received 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] corosync_ring_id_store Storing new sequence id for ring 1c Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_commit_enter entering COMMIT state. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token got commit token Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter entering RECOVERY state. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter position [0] member 192.168.0.4: Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter previous ring seq 18 rep 192.168.0.4 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter aru 0 high delivered 0 received flag 1 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter Did not need to originate any messages in recovery. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token got commit token Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token Sending initial ORF token Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token retrans flag count 4 token aru 0 install seq 0 aru 0 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] old_ring_state_reset Resetting old ring state Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] deliver_messages_from_recovery_to_regular recovery to regular 1-0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemudpu_member_set_active Marking UDPU member 192.168.0.4 active Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] member_object_joined Member joined: r(0) ip(192.168.0.4) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_operational_enter entering OPERATIONAL state. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [TOTEM ] memb_state_operational_enter A new membership (192.168.0.4:28) was formed. Members joined: 3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[3]: votes: 1, expected: 3 flags: 8 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: Yes Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=1, expected_votes=3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [SYNC ] sync_barrier_handler Committing synchronization for corosync configuration map access Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_sync_activate Single node sync -> no action Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [CPG ] downlist_log comparing: sender r(0) ip(192.168.0.4) ; members(old:0 left:0) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [CPG ] downlist_log chosen downlist: sender r(0) ip(192.168.0.4) ; members(old:0 left:0) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [SYNC ] sync_barrier_handler Committing synchronization for corosync cluster closed process group service v1.01 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: Yes Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_gather_enter entering GATHER state from 9(merge during operational state). Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_commit_token_create Creating commit token because I am the rep. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] old_ring_state_save Saving state aru 5 high seq received 5 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] corosync_ring_id_store Storing new sequence id for ring 20 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_commit_enter entering COMMIT state. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token got commit token Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter entering RECOVERY state. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter TRANS [0] member 192.168.0.4: Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter position [0] member 192.168.0.4: Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter previous ring seq 1c rep 192.168.0.4 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter aru 5 high delivered 5 received flag 1 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter position [1] member 192.168.0.5: Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter previous ring seq 1c rep 192.168.0.5 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter aru 4a high delivered 4a received flag 1 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter Did not need to originate any messages in recovery. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token got commit token Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token Sending initial ORF token Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token retrans flag count 4 token aru 0 install seq 0 aru 0 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] old_ring_state_reset Resetting old ring state Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] deliver_messages_from_recovery_to_regular recovery to regular 1-0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totemudpu_member_set_active Marking UDPU member 192.168.0.5 active Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [MAIN ] member_object_joined Member joined: r(0) ip(192.168.0.5) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_state_operational_enter entering OPERATIONAL state. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [TOTEM ] memb_state_operational_enter A new membership (192.168.0.4:32) was formed. Members joined: 2 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[3]: votes: 1, expected: 3 flags: 8 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: Yes Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=1, expected_votes=3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[0]: votes: 0, expected: 0 flags: 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [SYNC ] sync_barrier_handler barrier for old ring - discarding Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [SYNC ] sync_barrier_handler Committing synchronization for corosync configuration map access Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_sync_activate My config version is 0 -> no action Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_joinlist got joinlist message from node 0x2 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [CPG ] downlist_log comparing: sender r(0) ip(192.168.0.4) ; members(old:1 left:0) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [CPG ] downlist_log comparing: sender r(0) ip(192.168.0.5) ; members(old:1 left:0) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [CPG ] downlist_log chosen downlist: sender r(0) ip(192.168.0.5) ; members(old:1 left:0) Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [SYNC ] sync_barrier_handler Committing synchronization for corosync cluster closed process group service v1.01 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [CPG ] joinlist_inform_clients joinlist_messages[0] group:crmd\x00, ip:r(0) ip(192.168.0.5) , pid:7403 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [CPG ] joinlist_inform_clients joinlist_messages[1] group:attrd\x00, ip:r(0) ip(192.168.0.5) , pid:7369 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [CPG ] joinlist_inform_clients joinlist_messages[2] group:stonithd\x00, ip:r(0) ip(192.168.0.5) , pid:7346 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [CPG ] joinlist_inform_clients joinlist_messages[3] group:cib\x00, ip:r(0) ip(192.168.0.5) , pid:7337 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [CPG ] joinlist_inform_clients joinlist_messages[4] group:pacemakerd\x00, ip:r(0) ip(192.168.0.5) , pid:7234 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[3]: votes: 1, expected: 3 flags: 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=1, expected_votes=3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[0]: votes: 0, expected: 0 flags: 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 2 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[2]: votes: 1, expected: 3 flags: 1 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: Yes Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=2, expected_votes=3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 2 state=1, votes=1, expected=3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] get_lowest_node_id lowest node id: 2 us: 3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] get_highest_node_id highest node id: 3 us: 3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] are_we_quorate quorum regained, resuming activity Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 2 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[0]: votes: 0, expected: 0 flags: 0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [SYNC ] sync_barrier_handler Committing synchronization for corosync vote quorum service v1.0 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=2, expected_votes=3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 2 state=1, votes=1, expected=3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] get_lowest_node_id lowest node id: 2 us: 3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [VOTEQ ] get_highest_node_id highest node id: 3 us: 3 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [QUORUM] quorum_api_set_quorum This node is within the primary component and will provide service. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [QUORUM] log_view_list Members[2]: 3 2 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [QUORUM] send_library_notification sending quorum notification to (nil), length = 56 Feb 17 18:26:07 [5459] node-3.test.domain.local corosync notice [MAIN ] corosync_sync_completed Completed service synchronization, ready to provide service. Feb 17 18:26:07 [5459] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 0 Feb 17 18:26:13 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-18) Feb 17 18:26:13 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:13 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:13 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:13 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2643520 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-19) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2651c00 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-19) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-19) state:2 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2651c00 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-19-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-19-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-19-header Set r/w permissions for uid=108, gid=115 on /var/log/corosync.log Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: notice: mcp_read_config: Configured corosync to accept connections from group 115: OK (1) Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-18-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-18-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-18-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: notice: main: Starting Pacemaker 1.1.12 (Build: 561c4cf): generated-manpages agent-manpages ncurses libqb-logging libqb-ipc lha-fencing nagios heartbeat corosync-native snmp libesmtp acls Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: main: Maximum core file size is: 18446744073709551615 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-18) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-18) state:2 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2643520 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: qb_ipcs_us_publish: server name: pacemakerd Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-18-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-18-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-18-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-18) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: cluster_connect_cfg: Our nodeid: 3 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-19) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7fd9d2651c00, cpd=0x7fd9d2644c04 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-20) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d28536c0 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_procjoin got procjoin message from cluster node 0x3 (r(0) ip(192.168.0.4) ) for pid 7299 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-20-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-20-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-20-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: crm_get_peer: Created entry 818c3647-9e14-4a74-a0f2-f9d587c68ebe/0x14a02b0 for node (null)/3 (1 total) Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: crm_get_peer: Node 3 has uuid 3 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3] - corosync-cpg is now online Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: cluster_connect_quorum: Configuring Pacemaker to obtain quorum from Corosync Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-20) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-20) state:2 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d28536c0 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-20-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-20-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-20-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-20) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7fd9d2954050 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7fd9d2954050 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7fd9d2954050 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: notice: cluster_connect_quorum: Quorum acquired Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_trackstart got trackstart request on 0x7fd9d2954050 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_trackstart sending initial status to 0x7fd9d2954050 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QUORUM] send_library_notification sending quorum notification to 0x7fd9d2954050, length = 56 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-21) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2955020 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-22) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2956f30 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-22-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-22-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-22-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 1 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: crm_get_peer: Created entry 6baed6ad-b407-4e20-962e-473346c1a24e/0x14a1890 for node (null)/1 (2 total) Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: crm_get_peer: Node 1 has uuid 1 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-22) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-22) state:2 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2956f30 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-22-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-22-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-22-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-22) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2956f30 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-22-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-22-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-22-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-22) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-22) state:2 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2956f30 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-22-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-22-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-22-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-22) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2956f30 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-22-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-22-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-22-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: crm_get_peer: Created entry c01e0a83-3dbe-4d0b-9bde-ca36b3bbd494/0x14a1a80 for node (null)/2 (3 total) Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: crm_get_peer: Node 2 has uuid 2 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-22) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-22) state:2 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2956f30 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-22-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-22-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-22-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-21) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-21) state:2 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2955020 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-21) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2955020 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: crm_get_peer: Node 3 is now known as node-3.test.domain.local Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-21) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-21) state:2 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2955020 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-21) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2955020 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: start_child: Using uid=108 and group=115 for process cib Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: start_child: Forked child 7461 for process cib Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-21) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-21) state:2 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2955020 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:14 [7461] node-3.test.domain.local cib: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-21) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:14 [7461] node-3.test.domain.local cib: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:26:14 [7461] node-3.test.domain.local cib: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:26:14 [7461] node-3.test.domain.local cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.xml (digest: /var/lib/pacemaker/cib/cib.xml.sig) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2955020 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000000100 (was 00000000000000000000000004000000) Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: start_child: Forked child 7473 for process stonith-ng Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-21) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-21) state:2 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2955020 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:14 [7461] node-3.test.domain.local cib: info: validate_with_relaxng: Creating RNG parser context Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-21) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:14 [7473] node-3.test.domain.local stonithd: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/root Feb 17 18:26:14 [7473] node-3.test.domain.local stonithd: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:26:14 [7473] node-3.test.domain.local stonithd: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:26:14 [7473] node-3.test.domain.local stonithd: notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2955da0 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7473-22) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7473] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7fd9d2956500, cpd=0x7fd9d2956c54 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:14 [7473] node-3.test.domain.local stonithd: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000100100 (was 00000000000000000000000000000100) Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: start_child: Forked child 7495 for process lrmd Feb 17 18:26:14 [7495] node-3.test.domain.local pacemaker_remoted: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/root Feb 17 18:26:14 [7495] node-3.test.domain.local pacemaker_remoted: info: qb_ipcs_us_publish: server name: lrmd Feb 17 18:26:14 [7495] node-3.test.domain.local pacemaker_remoted: info: main: Starting Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-21) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-21) state:2 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2955da0 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-21) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7461] node-3.test.domain.local cib: debug: activateCibXml: Triggering CIB write for start op Feb 17 18:26:14 [7461] node-3.test.domain.local cib: info: startCib: CIB Initialization completed successfully Feb 17 18:26:14 [7461] node-3.test.domain.local cib: notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2955da0 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_procjoin got procjoin message from cluster node 0x3 (r(0) ip(192.168.0.4) ) for pid 7473 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7473-23) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7473] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957c00 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-24) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7fd9d2959950, cpd=0x7fd9d295a0a4 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000100110 (was 00000000000000000000000000100100) Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: start_child: Using uid=108 and group=115 for process attrd Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: info: start_child: Forked child 7543 for process attrd Feb 17 18:26:14 [7461] node-3.test.domain.local cib: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-21) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-21) state:2 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2955da0 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_procjoin got procjoin message from cluster node 0x3 (r(0) ip(192.168.0.4) ) for pid 7461 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-21) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:14 [7543] node-3.test.domain.local attrd: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster Feb 17 18:26:14 [7543] node-3.test.domain.local attrd: info: main: Starting up Feb 17 18:26:14 [7543] node-3.test.domain.local attrd: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:26:14 [7543] node-3.test.domain.local attrd: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:26:14 [7543] node-3.test.domain.local attrd: notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2955020 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-25) Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:14 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d295cab0 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-26) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7fd9d295e6a0, cpd=0x7fd9d295ef94 Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-23-header Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-23-header Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-23-header Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: info: crm_get_peer: Created entry 96f8feab-e0e1-4589-86f3-c73d3e20f607/0x2250300 for node (null)/3 (1 total) Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: info: crm_get_peer: Node 3 has uuid 3 Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3] - corosync-cpg is now online Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: info: init_cs_connection_once: Connection to 'corosync': established Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000101110 (was 00000000000000000000000000100110) Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: info: start_child: Using uid=108 and group=115 for process pengine Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: info: start_child: Forked child 7585 for process pengine Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7473-23) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7473-23) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957c00 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-23-header Feb 17 18:26:15 [7585] node-3.test.domain.local pengine: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster Feb 17 18:26:15 [7585] node-3.test.domain.local pengine: debug: main: Init server comms Feb 17 18:26:15 [7585] node-3.test.domain.local pengine: info: qb_ipcs_us_publish: server name: pengine Feb 17 18:26:15 [7585] node-3.test.domain.local pengine: info: main: Starting pengine Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-21) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-21) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2955020 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_procjoin got procjoin message from cluster node 0x3 (r(0) ip(192.168.0.4) ) for pid 7543 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7473-21) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7473] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957c00 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-25-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-25-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-25-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: crm_get_peer: Created entry 6e07e412-b494-4dc2-9c0a-ba550faa105f/0xb4be70 for node (null)/3 (1 total) Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: crm_get_peer: Node 3 has uuid 3 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3] - corosync-cpg is now online Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: init_cs_connection_once: Connection to 'corosync': established Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-23) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29587c0 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-25) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-25) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d295cab0 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-25-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-25-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-25-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-25) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d295cab0 Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-21-header Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-21-header Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-21-header Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: info: crm_get_peer: Node 3 is now known as node-3.test.domain.local Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-23-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-23-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-23-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000111110 (was 00000000000000000000000000101110) Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: info: start_child: Using uid=108 and group=115 for process crmd Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: info: start_child: Forked child 7621 for process crmd Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7621] node-3.test.domain.local crmd: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster Feb 17 18:26:15 [7621] node-3.test.domain.local crmd: notice: main: CRM Git Version: 561c4cf Feb 17 18:26:15 [7621] node-3.test.domain.local crmd: debug: crmd_init: Starting crmd Feb 17 18:26:15 [7621] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_STARTUP: [ state=S_STARTING cause=C_STARTUP origin=crmd_init ] Feb 17 18:26:15 [7621] node-3.test.domain.local crmd: info: do_log: FSA: Input I_STARTUP from crmd_init() received in state S_STARTING Feb 17 18:26:15 [7621] node-3.test.domain.local crmd: debug: do_startup: Registering Signal Handlers Feb 17 18:26:15 [7621] node-3.test.domain.local crmd: debug: do_startup: Creating CIB and LRM objects Feb 17 18:26:15 [7621] node-3.test.domain.local crmd: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:26:15 [7621] node-3.test.domain.local crmd: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:26:15 [7621] node-3.test.domain.local crmd: info: crm_ipc_connect: Could not establish cib_shm connection: Connection refused (111) Feb 17 18:26:15 [7621] node-3.test.domain.local crmd: debug: cib_native_signon_raw: Connection unsuccessful (0 (nil)) Feb 17 18:26:15 [7621] node-3.test.domain.local crmd: debug: cib_native_signon_raw: Connection to CIB failed: Transport endpoint is not connected Feb 17 18:26:15 [7621] node-3.test.domain.local crmd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5ba90 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7473-21) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7473-21) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957c00 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-23) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-23) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29587c0 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7473-21) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7473] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957c00 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-23) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957030 Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-21-header Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-21-header Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-21-header Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: info: crm_ipc_connect: Could not establish cib_rw connection: Connection refused (111) Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: cib_native_signon_raw: Connection unsuccessful (0 (nil)) Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: cib_native_signon_raw: Connection to CIB failed: Transport endpoint is not connected Feb 17 18:26:15 [7473] node-3.test.domain.local stonithd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7473-21) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7473-21) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957c00 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-21-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-25-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-25-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-25-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: crm_get_peer: Created entry efcf4caf-8dc3-40af-940a-bc033d0c8e45/0x1b9b8d0 for node (null)/3 (1 total) Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: crm_get_peer: Node 3 has uuid 3 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3] - corosync-cpg is now online Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: init_cs_connection_once: Connection to 'corosync': established Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: crm_get_peer: Node 3 is now known as node-3.test.domain.local Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-25) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-25) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d295cab0 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-25-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-25-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-25-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-23-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-23-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-23-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000111310 (was 00000000000000000000000000111110) Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: info: main: Starting mainloop Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: info: pcmk_quorum_notification: Membership 32: quorum retained (2) Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: pcmk_quorum_notification: Member[0] 3 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-21) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957c00 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5ba90 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-23) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-23) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957030 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957030 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-25) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d295d8a0 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: qb_ipcs_us_publish: server name: cib_ro Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: qb_ipcs_us_publish: server name: cib_rw Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: qb_ipcs_us_publish: server name: cib_shm Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: cib_init: Starting cib mainloop Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: pcmk_cpg_membership: Joined[0.0] cib.3 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957030 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-25-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-25-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-25-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: notice: crm_update_peer_state: pcmk_quorum_notification: Node node-3.test.domain.local[3] - state is now member (was (null)) Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: pcmk_quorum_notification: Member[1] 2 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-21-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-21-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-21-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: crm_get_peer: Node 3 is now known as node-3.test.domain.local Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957030 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-25) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-25) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d295d8a0 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-25-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-25-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-25-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-21) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-21) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957c00 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-21) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957c00 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-25) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2958c90 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: crm_get_peer: Created entry 5e4129e2-65e8-40e8-b3b4-f5c65d23930a/0xb4f140 for node (null)/2 (2 total) Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: crm_get_peer: Node 2 has uuid 2 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: pcmk_cpg_membership: Member[0.0] cib.2 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[2] - corosync-cpg is now online Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957030 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: info: pcmk_quorum_notification: Obtaining name for new node 2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957030 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-21) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-21) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957c00 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-25-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-25-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-25-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: main: Cluster connection active Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: qb_ipcs_us_publish: server name: attrd Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: main: Accepting attribute updates Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: notice: main: Starting mainloop... Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: pcmk_cpg_membership: Joined[0.0] attrd.3 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-21) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957c00 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-25) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-25) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2958c90 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-25-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-25-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-25-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: pcmk_cpg_membership: Member[0.1] cib.3 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: child_waitpid: wait(7720) = 0: Resource temporarily unavailable (11) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-25) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-87.raw Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: write_cib_contents: Writing CIB to disk Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: write_cib_contents: Wrote version 0.155.0 of the CIB to disk (digest: f05934746551e8386bf033d8f8760fc6) Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: write_cib_contents: Wrote digest f05934746551e8386bf033d8f8760fc6 to disk Feb 17 18:26:15 [7461] node-3.test.domain.local cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.RYrTQB (digest: /var/lib/pacemaker/cib/cib.5dSHob) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7461] node-3.test.domain.local cib: debug: write_cib_contents: Activating /var/lib/pacemaker/cib/cib.RYrTQB Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d295ddf0 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957030 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-21) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-21) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957c00 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-21) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957c00 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: notice: crm_update_peer_state: pcmk_quorum_notification: Node (null)[2] - state is now member (was (null)) Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: info: pcmk_quorum_notification: State of node (null)[1] is still unknown Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: info: crm_get_peer: Node 2 is now known as node-2.test.domain.local Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-21) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-21) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-25-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957c00 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-25-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-25-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: crm_get_peer: Created entry a1960322-e33f-419b-8e67-3b55452eedc5/0x1b9d170 for node (null)/2 (2 total) Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: crm_get_peer: Node 2 has uuid 2 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: pcmk_cpg_membership: Member[0.0] attrd.2 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[2] - corosync-cpg is now online Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-25) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-25) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d295ddf0 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-25-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-25-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-25-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7299-21) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7299] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957c00 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d295c9b0 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-2.test.domain.local now has process list: 00000000000000000000000000111310 (was 00000000000000000000000000000000) Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(7461) = 0: Resource temporarily unavailable (11) Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(7473) = 0: Resource temporarily unavailable (11) Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(7495) = 0: Resource temporarily unavailable (11) Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(7543) = 0: Resource temporarily unavailable (11) Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(7585) = 0: Resource temporarily unavailable (11) Feb 17 18:26:15 [7299] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(7621) = 0: Resource temporarily unavailable (11) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7299-21) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-21) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957c00 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7299-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7299-21-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7299-21-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:15 [7543] node-3.test.domain.local attrd: info: pcmk_cpg_membership: Member[0.1] attrd.3 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d295c9b0 Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:15 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0xb4fe70 for uid=108 gid=115 pid=7621 id=11c8a8ce-2dd9-4a89-85b7-0e7b4e517bd1 Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (7461-7621-11) Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [7621] Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: cib_common_callback_worker: Setting cib_refresh_notify callbacks for crmd (11c8a8ce-2dd9-4a89-85b7-0e7b4e517bd1): on Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: cib_common_callback_worker: Setting cib_diff_notify callbacks for crmd (11c8a8ce-2dd9-4a89-85b7-0e7b4e517bd1): on Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: do_cib_control: CIB connection established Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-21) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7fd9d2957c00, cpd=0x7fd9d29577c4 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d295c9b0 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_procjoin got procjoin message from cluster node 0x3 (r(0) ip(192.168.0.4) ) for pid 7621 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: crm_get_peer: Created entry 6e67a1ed-573f-44d3-81cc-c9ee46df128d/0x273a610 for node (null)/3 (1 total) Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: crm_get_peer: Node 3 has uuid 3 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3] - corosync-cpg is now online Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: init_cs_connection_once: Connection to 'corosync': established Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d295c9b0 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957150 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: crm_get_peer: Node 3 is now known as node-3.test.domain.local Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: peer_update_callback: node-3.test.domain.local is now (null) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957150 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0xbd0e10 for uid=0 gid=0 pid=7473 id=ecbe107a-373d-4ead-b604-8f46a39ad9cc Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (7461-7473-12) Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [7473] Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: cib_common_callback_worker: Setting cib_diff_notify callbacks for crmd (ecbe107a-373d-4ead-b604-8f46a39ad9cc): on Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: notice: setup_cib: Watching for stonith topology changes Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: qb_ipcs_us_publish: server name: stonith-ng Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: main: Starting stonithd mainloop Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: pcmk_cpg_membership: Joined[0.0] stonithd.3 Feb 17 18:26:16 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957150 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7473-25) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7473] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5c7a0 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-25-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-25-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-25-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: crm_get_peer: Created entry 29b758bf-ced8-406c-a58f-512f425d43c6/0x22523f0 for node (null)/2 (2 total) Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: crm_get_peer: Node 2 has uuid 2 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: pcmk_cpg_membership: Member[0.0] stonithd.2 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[2] - corosync-cpg is now online Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: st_peer_update_callback: Broadcasting our uname because of node 2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7473-25) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7473-25) state:2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: cluster_connect_quorum: Configuring Pacemaker to obtain quorum from Corosync Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5c7a0 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-25-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-25-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-25-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957150 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7473-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7473] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957150 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-25) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-23-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-23-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-23-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: notice: cluster_connect_quorum: Quorum acquired Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7fd9d295dbf0 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7fd9d295dbf0 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7fd9d295dbf0 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7473-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7473-23) state:2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957150 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-23-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_trackstart got trackstart request on 0x7fd9d295dbf0 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_trackstart sending initial status to 0x7fd9d295dbf0 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QUORUM] send_library_notification sending quorum notification to 0x7fd9d295dbf0, length = 56 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7473-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7473] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2957150 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-27) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29587c0 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-23-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-23-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-23-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: pcmk_cpg_membership: Member[0.1] stonithd.3 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: init_cib_cache_cb: Updating device list from the cib: init Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: cib_devices_update: Updating devices to version 0.155.0 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: unpack_config: Stop all active resources: false Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: unpack_config: Default stickiness: 0 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: apply_system_health: Applying automated node health strategy: migrate-on-red Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: apply_system_health: Node node-1.test.domain.local has an combined system health of 0 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: apply_system_health: Node node-2.test.domain.local has an combined system health of 0 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: apply_system_health: Node node-3.test.domain.local has an combined system health of 0 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: info: crm_get_peer: Node 2 is now known as node-2.test.domain.local Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: st_peer_update_callback: Broadcasting our uname because of node 2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7473-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7473-23) state:2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2957150 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7473-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7473] Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5be30 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-23-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-23-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7473-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7473-23) state:2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5be30 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb80 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7473-28) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7473] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-28-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-28-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-28-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 1 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: crm_get_peer: Created entry f5bd92e7-877f-4a1b-a048-92792e53eec6/0x293c580 for node (null)/1 (2 total) Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: crm_get_peer: Node 1 has uuid 1 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d295a1e0 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7473-28) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7473-28) state:2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d295a1e0 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-28-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-28-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-28-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb80 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7473-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7473] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d295a1e0 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-23-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-23-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7473-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7473-23) state:2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d295a1e0 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-23-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7473-23) Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-23-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-23-header Feb 17 18:26:16 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7473] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb80 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7473-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7473-23) state:2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb80 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7473-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7473-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7473-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb80 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb80 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb80 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: crm_get_peer: Created entry 0aee89c4-9d8b-44b3-964c-423ce5cf0ef2/0x293c820 for node (null)/2 (3 total) Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: crm_get_peer: Node 2 has uuid 2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb80 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-27-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-27-header Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-27-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-27) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-27) state:2 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29587c0 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-27-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-27-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-27-header Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb50 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-27) Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:16 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-27-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-27-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-27-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 1 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-27) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-27) state:2 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-27-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-27-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-27-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-27) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-27-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-27-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-27-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-27) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-27) state:2 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-27-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-27-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-27-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-27) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-27-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-27-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-27-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-27) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-27) state:2 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-27-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-27-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-27-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: do_ha_control: Connected to the cluster Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: do_lrm_control: Connecting to the LRM Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: lrmd_ipc_connect: Connecting to lrmd Feb 17 18:26:17 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section nodes to master (origin=local/crmd/3) Feb 17 18:26:17 [7495] node-3.test.domain.local pacemaker_remoted: debug: crm_client_new: Connecting 0x1796b10 for uid=108 gid=115 pid=7621 id=5488a65d-580e-45a9-9600-4c95e6cd32de Feb 17 18:26:17 [7495] node-3.test.domain.local pacemaker_remoted: debug: handle_new_connection: IPC credentials authenticated (7495-7621-7) Feb 17 18:26:17 [7495] node-3.test.domain.local pacemaker_remoted: debug: qb_ipcs_shm_connect: connecting to client [7621] Feb 17 18:26:17 [7495] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:17 [7495] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:17 [7495] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:17 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed register operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: do_lrm_control: LRM connection established Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: do_started: Delaying start, no membership data (0000000000100000) Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: register_fsa_input_adv: Stalling the FSA pending further input: source=do_started cause=C_FSA_INTERNAL data=(nil) queue=0 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: s_crmd_fsa: Exiting the FSA: queue=0, fsa_actions=0x2, stalled=true Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: pcmk_quorum_notification: Membership 32: quorum retained (2) Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: pcmk_quorum_notification: Member[0] 3 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb50 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-27) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bbd0 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-27-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-27-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-27-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: notice: crm_update_peer_state: pcmk_quorum_notification: Node node-3.test.domain.local[3] - state is now member (was (null)) Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: peer_update_callback: node-3.test.domain.local is now member (was (null)) Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: pcmk_quorum_notification: Member[1] 2 Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-27) Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-27) state:2 Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:17 [7461] node-3.test.domain.local cib: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:17 [7461] node-3.test.domain.local cib: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bbd0 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-27-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-27-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-27-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: pcmk_quorum_notification: Obtaining name for new node 2 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb50 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:17 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-3.test.domain.local/crmd/3, version=0.155.0) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb50 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: notice: crm_update_peer_state: pcmk_quorum_notification: Node (null)[2] - state is now member (was (null)) Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: pcmk_quorum_notification: State of node (null)[1] is still unknown Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: post_cache_update: Updated cache after membership event 32. Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: post_cache_update: post_cache_update added action A_ELECTION_CHECK to the FSA Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: do_started: Delaying start, Config not read (0000000000000040) Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: register_fsa_input_adv: Stalling the FSA pending further input: source=do_started cause=C_FSA_INTERNAL data=(nil) queue=0 Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: s_crmd_fsa: Exiting the FSA: queue=0, fsa_actions=0x200000002, stalled=true Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: config_query_callback: Call 4 : Parsing CIB options Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: config_query_callback: Shutdown escalation occurs after: 1200000ms Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: config_query_callback: Checking for expired actions every 190000ms Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: do_started: Init server comms Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: qb_ipcs_us_publish: server name: crmd Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: notice: do_started: The local CRM is operational Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: do_election_check: Ignore election check: we not in an election Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_STARTING cause=C_FSA_INTERNAL origin=do_started ] Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: info: do_log: FSA: Input I_PENDING from do_started() received in state S_STARTING Feb 17 18:26:17 [7621] node-3.test.domain.local crmd: notice: do_state_transition: State transition S_STARTING -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_started ] Feb 17 18:26:17 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: do_cl_join_query: Querying for a DC Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=19 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: info: pcmk_cpg_membership: Joined[0.0] crmd.3 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: info: pcmk_cpg_membership: Member[0.0] crmd.2 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[2] - corosync-cpg is now online Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: info: pcmk_cpg_membership: Member[0.1] crmd.3 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bdf0 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bdf0 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bda0 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: info: crm_get_peer: Node 2 is now known as node-2.test.domain.local Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: info: peer_update_callback: node-2.test.domain.local is now member Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bda0 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bda0 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bda0 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bda0 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: handle_request: Raising I_JOIN_OFFER: join-2 Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ] Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: info: update_dc: Set DC to node-2.test.domain.local (3.0.9) Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA Feb 17 18:26:18 [7621] node-3.test.domain.local crmd: debug: te_connect_stonith: Attempting connection to fencing daemon... Feb 17 18:26:18 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bda0 Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:18 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: crm_client_new: Connecting 0x24c2800 for uid=108 gid=115 pid=7621 id=8b9e87b6-108b-4fec-b999-0b77c2a0c943 Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: handle_new_connection: IPC credentials authenticated (7473-7621-10) Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: qb_ipcs_shm_connect: connecting to client [7621] Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: stonith_command: Processing register 10 from crmd.7621 ( 0) Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: stonith_command: Processed register from crmd.7621: OK (0) Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: stonith_api_signon: Connection to STONITH successful Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: stonith_command: Processing st_notify 11 from crmd.7621 ( 0) Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: handle_request: Setting st_notify_disconnect callbacks for crmd.7621 (8b9e87b6-108b-4fec-b999-0b77c2a0c943): ON Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: stonith_command: Processed st_notify from crmd.7621: OK (0) Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: stonith_command: Processing st_notify 12 from crmd.7621 ( 0) Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: handle_request: Setting st_notify_fence callbacks for crmd.7621 (8b9e87b6-108b-4fec-b999-0b77c2a0c943): ON Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: stonith_command: Processed st_notify from crmd.7621: OK (0) Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: join_query_callback: Respond to join offer join-2 from node-2.test.domain.local Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bda0 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bda0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: crm_get_peer: Node 2 is now known as node-2.test.domain.local Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_process_replace: Digest matched on replace from node-2.test.domain.local: b821c8b2176bc364a9194d464ecc0a2b Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_process_replace: Replaced 0.155.0 with 0.155.380 from node-2.test.domain.local Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.0 2 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.380 (null) Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=380, @dc-uuid=2 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status: Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status: Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: activateCibXml: Triggering CIB write for cib_replace op Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_replace_notify: Replaced: 0.155.0 -> 0.155.380 from node-2.test.domain.local Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_replace operation for section 'all': OK (rc=0, origin=node-2.test.domain.local/crmd/248, version=0.155.380) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.380 to 0.155.0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29587c0 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: handle_request: Raising I_JOIN_RESULT: join-2 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ] Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: do_cl_join_finalize_respond: Confirming join join-2: join_ack_nack Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-2.test.domain.local/crmd/249, version=0.155.380) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29587c0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29587c0 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: do_cl_join_finalize_respond: join-2: Join complete. Sending local LRM status to node-2.test.domain.local Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: info: erase_status_tag: Deleting xpath: //node_state[@uname='node-3.test.domain.local']/transient_attributes Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: info: update_attrd_helper: Connecting to attrd... 5 retries remaining Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x1b9dd70 for uid=108 gid=115 pid=7621 id=4cc0df8f-632b-4df4-a068-d6edb9ee414f Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (7543-7621-9) Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7621] Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: attrd_update_delegate: Sent update: terminate=(null) for node-3.test.domain.local Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: attrd_update_delegate: Sent update: shutdown=0 for node-3.test.domain.local Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crmd: terminate= Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for terminate Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null) Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crmd: shutdown=0 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for shutdown Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: (null), Stored: (null) Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of shutdown is 0 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: shutdown (0) Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation shutdown=0: cib not connected Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-2.test.domain.local/crmd/250, version=0.155.380) Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_delete operation for section //node_state[@uname='node-3.test.domain.local']/transient_attributes to master (origin=local/crmd/7) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-88.raw Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: write_cib_contents: Writing CIB to disk Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: write_cib_contents: Wrote version 0.155.0 of the CIB to disk (digest: 70331ce3b0d5b061b5f74dcab7ba23ca) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29587c0 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: write_cib_contents: Wrote digest 70331ce3b0d5b061b5f74dcab7ba23ca to disk Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.q0MvsI (digest: /var/lib/pacemaker/cib/cib.OHzmIw) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: write_cib_contents: Activating /var/lib/pacemaker/cib/cib.q0MvsI Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29587c0 Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ] Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: info: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: do_cib_replaced: Updating the CIB after a replace: DC=false Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: notice: attrd_local_callback: Sending full refresh (origin=crmd) Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: shutdown (0) Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation shutdown=0: cib not connected Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29587c0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: //node_state[@uname='node-3.test.domain.local']/transient_attributes was already removed Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='node-3.test.domain.local']/transient_attributes: OK (rc=0, origin=node-3.test.domain.local/crmd/7, version=0.155.380) Feb 17 18:26:19 [7621] node-3.test.domain.local crmd: debug: erase_xpath_callback: Deletion of "//node_state[@uname='node-3.test.domain.local']/transient_attributes": OK (rc=0) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29587c0 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_delete op for //node_state[@uname='node-2.test.domain.local']/lrm (/cib/status/node_state[1]/lrm) Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.380 2 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.381 (null) Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: -- /cib/status/node_state[@id='2']/lrm[@id='2'] Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=381 Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.381 to 0.155.380 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='node-2.test.domain.local']/lrm: OK (rc=0, origin=node-2.test.domain.local/crmd/251, version=0.155.381) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29587c0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29587c0 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.381 2 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.382 (null) Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=382 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_lrm_query_internal Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']: Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29587c0 Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.382 to 0.155.381 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/252, version=0.155.382) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/563, version=0.155.382) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: info: crm_get_peer: Node 2 is now known as node-2.test.domain.local Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29587c0 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/565, version=0.155.382) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for rabbit_get_alarms_timeouts Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation rabbit_get_alarms_timeouts=: cib not connected Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29587c0 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29808f0 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: //node_state[@uname='node-3.test.domain.local']/lrm was already removed Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='node-3.test.domain.local']/lrm: OK (rc=0, origin=node-2.test.domain.local/crmd/253, version=0.155.382) Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29808f0 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for fail-count-p_mysql Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation fail-count-p_mysql=: cib not connected Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29587c0 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.382 2 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.383 (null) Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=383 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']: @crmd=online, @crm-debug-origin=do_lrm_query_internal, @uname=node-3.test.domain.local, @join=member, @expected=member Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']: Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/254, version=0.155.383) Feb 17 18:26:19 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.383 to 0.155.382 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for free_swap Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation free_swap=: cib not connected Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29587c0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29587c0 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:19 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-2.test.domain.local/crmd/257, version=0.155.383) Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for last-failure-p_neutron-l3-agent Feb 17 18:26:19 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation last-failure-p_neutron-l3-agent=: cib not connected Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29587c0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:19 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29587c0 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.383 2 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.384 (null) Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=384 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']: @crm-debug-origin=do_state_transition Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/258, version=0.155.384) Feb 17 18:26:20 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.384 to 0.155.383 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for gtid Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation gtid=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29587c0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29587c0 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section cib: OK (rc=0, origin=node-2.test.domain.local/crmd/259, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for rabbit_list_channels_timeouts Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation rabbit_list_channels_timeouts=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29587c0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29587c0 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/567, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for fail-count-p_neutron-l3-agent Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation fail-count-p_neutron-l3-agent=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29587c0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2959010 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/569, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for cpu_cores Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation cpu_cores=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2959010 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29587c0 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for arch Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation arch=: cib not connected Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/571, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29587c0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d295a1e0 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for ram_total Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation ram_total=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/573, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d295a1e0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d295a1e0 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for ram_free Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation ram_free=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/575, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d295a1e0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d295a1e0 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for rabbit_list_queues_timeouts Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation rabbit_list_queues_timeouts=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/577, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d295a1e0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d295a1e0 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for cpu_load Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation cpu_load=: cib not connected Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/579, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d295a1e0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5be30 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for root_free Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation root_free=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/581, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5be30 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5be30 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for var_lib_mysql_free Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation var_lib_mysql_free=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/583, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5be30 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5be30 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for last-failure-p_mysql Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation last-failure-p_mysql=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/585, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5be30 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5be30 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for cpu_info Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation cpu_info=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/587, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5be30 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5be30 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for os Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation os=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/589, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5be30 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5be30 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation shutdown=0: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/591, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5be30 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5be30 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for master-p_rabbitmq-server Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation master-p_rabbitmq-server=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/593, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5be30 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5be30 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/595, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5be30 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for master-p_conntrackd Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation master-p_conntrackd=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/597, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for cpu_speed Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation cpu_speed=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/599, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for probe_complete Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation probe_complete=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/601, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for var_log_free Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation var_log_free=: cib not connected Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/603, version=0.155.384) Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/605, version=0.155.384) Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/607, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/609, version=0.155.384) Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/611, version=0.155.384) Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/613, version=0.155.384) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb50 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: cib_connect: CIB signon attempt 1 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0xe39950 for uid=108 gid=115 pid=7543 id=6556cada-a709-465a-8d83-ee45a3a680c7 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (7461-7543-13) Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [7543] Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: cib_connect: Connected to the CIB after 1 signon attempts Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_common_callback_worker: Setting cib_refresh_notify callbacks for attrd (6556cada-a709-465a-8d83-ee45a3a680c7): on Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: info: cib_connect: Sending full refresh now that we're connected to the cib Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit_get_alarms_timeouts'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for rabbit_get_alarms_timeouts=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='last-failure-p_neutron-l3-agent'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for last-failure-p_neutron-l3-agent=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='free_swap'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for free_swap=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for gtid=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit_list_channels_timeouts'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for rabbit_list_channels_timeouts=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='fail-count-p_neutron-l3-agent'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for fail-count-p_neutron-l3-agent=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_cores'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for cpu_cores=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='arch'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for arch=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_total'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for ram_total=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='terminate'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for terminate=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for ram_free=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit_list_queues_timeouts'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for rabbit_list_queues_timeouts=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for cpu_load=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for root_free=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_lib_mysql_free'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for var_lib_mysql_free=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='last-failure-p_mysql'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for last-failure-p_mysql=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_info'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for cpu_info=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='os'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for os=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='shutdown'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 21: shutdown=0 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/21) Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='fail-count-p_mysql'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for fail-count-p_mysql=(null) passed Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.384 2 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.385 (null) Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=385 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']: Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/21, version=0.155.385) Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_rabbitmq-server'] does not exist Feb 17 18:26:20 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.385 to 0.155.384 Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for master-p_rabbitmq-server=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_speed'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for cpu_speed=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_conntrackd'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for master-p_conntrackd=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='probe_complete'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for probe_complete=(null) passed Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] does not exist Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for var_log_free=(null) passed Feb 17 18:26:20 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 21 for shutdown=0 passed Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'sysinfo_node-1.test.domain.local' not found (0 active resources) Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'sysinfo_node-1.test.domain.local' to the rsc list (1 active resources) Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=5:68:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=sysinfo_node-1.test.domain.local_monitor_0 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=5, reply=1, notify=0, exit=4202464 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:sysinfo_node-1.test.domain.local action:monitor call_id:5 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9017) = 0: Resource temporarily unavailable (11) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-1.test.domain.local_monitor_0:9017 - exited with rc=7 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: sysinfo_node-1.test.domain.local_monitor_0:9017:stderr [ rm: cannot remove '/var/run//SysInfo-sysinfo_node-1.test.domain.local': No such file or directory ] Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-1.test.domain.local_monitor_0:9017:stdout [ -- empty -- ] Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-1.test.domain.local action:monitor call_id:5 pid:9017 exit-code:7 exec-time:10ms queue-time:0ms Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bda0 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bda0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bda0 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_vrouter' not found (1 active resources) Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_vrouter:1' not found (1 active resources) Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_vrouter' to the rsc list (2 active resources) Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=6:68:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_vrouter_monitor_0 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=10, reply=1, notify=0, exit=4202464 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_vrouter action:monitor call_id:10 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource sysinfo_node-1.test.domain.local after monitor op complete (interval=0) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bda0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: info: services_os_action_execute: Managed SysInfo_meta-data_0 process 9024 exited with rc=0 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/8) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation sysinfo_node-1.test.domain.local_monitor_0: not running (node=node-3.test.domain.local, call=5, rc=7, cib-update=8, confirmed=true) Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: node-3.test.domain.local-sysinfo_node-1.test.domain.local_monitor_0:5 [ rm: cannot remove '/var/run//SysInfo-sysinfo_node-1.test.domain.local': No such file or directory\n ] Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'sysinfo_node-1.test.domain.local' with monitor op Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bda0 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.385 2 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.386 (null) Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=386 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']: @crm-debug-origin=do_update_resource Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/8, version=0.155.386) Feb 17 18:26:20 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.386 to 0.155.385 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bda0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-ns_vrouter: 2016/02/17_18:26:20 DEBUG: p_vrouter: check_ns(): recieved netns list: Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'vip__management' not found (2 active resources) Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'vip__management' to the rsc list (3 active resources) Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=7:68:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=vip__management_monitor_0 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=14, reply=1, notify=0, exit=4202464 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:vip__management action:monitor call_id:14 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9026) = 0: Resource temporarily unavailable (11) Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__management_monitor_0:9040 - exited with rc=7 Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__management_monitor_0:9040:stderr [ -- empty -- ] Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__management_monitor_0:9040:stdout [ -- empty -- ] Feb 17 18:26:20 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__management action:monitor call_id:14 pid:9040 exit-code:7 exec-time:28ms queue-time:0ms Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource vip__management after monitor op complete (interval=0) Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_IPaddr2_meta-data_0 process 9051 exited with rc=0 Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation vip__management_monitor_0: not running (node=node-3.test.domain.local, call=14, rc=7, cib-update=9, confirmed=true) Feb 17 18:26:20 [7621] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'vip__management' with monitor op Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/9) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-ns_vrouter: 2016/02/17_18:26:20 DEBUG: p_vrouter: get_ns(): added netns vrouter and set up lo Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:20 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:20 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.386 2 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.387 (null) Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=387 Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:26:20 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:21 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:21 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:21 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ word_size:263168 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:21 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:21 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:21 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:21 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:21 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:21 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.390 2 Feb 17 18:26:21 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.391 (null) Feb 17 18:26:21 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=391 Feb 17 18:26:21 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:26:21 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:21 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:21 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/14, version=0.155.392) Feb 17 18:26:21 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.392 to 0.155.391 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:21 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:21 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:21 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:21 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5be30 Feb 17 18:26:21 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:21 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:21 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:21 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:21 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_mysql' not found (8 active resources) Feb 17 18:26:21 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:21 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_mysql:1' not found (8 active resources) Feb 17 18:26:21 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:21 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_mysql' to the rsc list (9 active resources) Feb 17 18:26:21 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:26:21 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:21 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=13:68:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_mysql_monitor_0 Feb 17 18:26:21 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=41, reply=1, notify=0, exit=4202464 Feb 17 18:26:21 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_mysql action:monitor call_id:41 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5be30 Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:21 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:26 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:26 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:26 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:26 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:26 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:26 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:26 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:26 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:26 [7461] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 360e9dbd9ffdc85cef6daa64bf5e640f for 0.155.392 (0xb28ff0 0) Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:26 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:26 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:26 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:26 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:26 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:26 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:26 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:26 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header ocf-mysql-wss: 2016/02/17_18:26:26 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 2 retries left ocf-ns_vrouter: 2016/02/17_18:26:31 INFO: net.ipv4.conf.vr-host-base.rp_filter = 2 ocf-ns_vrouter: 2016/02/17_18:26:31 INFO: net.ipv4.conf.all.rp_filter = 2 ocf-ns_vrouter: 2016/02/17_18:26:31 DEBUG: Adding route on the host system to 10.109.0.0/24: 240.0.0.6 ocf-ns_vrouter: 2016/02/17_18:26:31 DEBUG: Adding route on the host system to 192.168.2.0/24: 240.0.0.6 ocf-ns_vrouter: 2016/02/17_18:26:31 DEBUG: Adding route on the host system to 192.168.1.0/24: 240.0.0.6 ocf-ns_vrouter: 2016/02/17_18:26:31 DEBUG: Adding route on the host system to 192.168.0.0/24: 240.0.0.6 ocf-ns_vrouter: 2016/02/17_18:26:31 DEBUG: Adding route on the host system to 10.109.1.0/24: 240.0.0.6 Feb 17 18:26:31 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_0:9026 - exited with rc=0 Feb 17 18:26:31 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_0:9026:stderr [ -- empty -- ] Feb 17 18:26:31 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_0:9026:stdout [ -- empty -- ] Feb 17 18:26:31 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:10 pid:9026 exit-code:0 exec-time:11000ms queue-time:0ms Feb 17 18:26:31 [7621] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_vrouter after monitor op complete (interval=0) Feb 17 18:26:31 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10195) = 0: Resource temporarily unavailable (11) Feb 17 18:26:31 [7621] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_vrouter_meta-data_0 process 10341 exited with rc=0 Feb 17 18:26:31 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/15) Feb 17 18:26:31 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_vrouter_monitor_0: ok (node=node-3.test.domain.local, call=10, rc=0, cib-update=15, confirmed=true) Feb 17 18:26:31 [7621] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_vrouter' with monitor op Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:31 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:31 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:31 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:31 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:31 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:31 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:31 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:31 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:31 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.392 2 Feb 17 18:26:31 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.393 (null) Feb 17 18:26:31 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=393 Feb 17 18:26:31 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:26:31 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:31 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/15, version=0.155.393) Feb 17 18:26:31 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.393 to 0.155.392 Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:31 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header ocf-mysql-wss: 2016/02/17_18:26:31 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 1 retries left Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 397263a31116b4fe255465403c668866 for 0.155.393 (0xb28ff0 0) Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header ocf-mysql-wss: 2016/02/17_18:26:36 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:26:36 INFO: MySQL is not running Feb 17 18:26:36 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_0:10195 - exited with rc=7 Feb 17 18:26:36 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_0:10195:stderr [ -- empty -- ] Feb 17 18:26:36 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_0:10195:stdout [ -- empty -- ] Feb 17 18:26:36 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_mysql action:monitor call_id:41 pid:10195 exit-code:7 exec-time:15040ms queue-time:0ms Feb 17 18:26:36 [7621] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_mysql after monitor op complete (interval=0) Feb 17 18:26:36 [7621] node-3.test.domain.local crmd: info: services_os_action_execute: Managed mysql-wss_meta-data_0 process 10390 exited with rc=0 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/16) Feb 17 18:26:36 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_mysql_monitor_0: not running (node=node-3.test.domain.local, call=41, rc=7, cib-update=16, confirmed=true) Feb 17 18:26:36 [7621] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_mysql' with monitor op Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:36 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:36 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.393 2 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.394 (null) Feb 17 18:26:36 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=394 Feb 17 18:26:36 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:26:36 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:36 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/16, version=0.155.394) Feb 17 18:26:36 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.394 to 0.155.393 Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:36 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=18:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_vrouter_monitor_30000 Feb 17 18:26:37 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=42, reply=1, notify=0, exit=4202464 Feb 17 18:26:37 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_vrouter action:monitor call_id:42 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for vip__vrouter_pub_start_0 Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=27:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=vip__vrouter_pub_start_0 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:37 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=43, reply=1, notify=0, exit=4202464 Feb 17 18:26:37 [7495] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:vip__vrouter_pub action:start call_id:43 ocf-ns_vrouter: 2016/02/17_18:26:37 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:26:37 INFO: net.ipv4.ip_forward = 1 Feb 17 18:26:37 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:10396 - exited with rc=0 Feb 17 18:26:37 [7495] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_vrouter_monitor_30000:10396:stderr [ Dump was interrupted and may be inconsistent. ] Feb 17 18:26:37 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:10396:stdout [ -- empty -- ] Feb 17 18:26:37 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:42 pid:10396 exit-code:0 exec-time:60ms queue-time:0ms Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_vrouter after monitor op complete (interval=30000) Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_vrouter_monitor_30000: ok (node=node-3.test.domain.local, call=42, rc=0, cib-update=17, confirmed=false) Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_vrouter_monitor_30000:42 [ Dump was interrupted and may be inconsistent.\n ] Feb 17 18:26:37 [7621] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_vrouter' with monitor op Feb 17 18:26:37 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10399) = 0: Resource temporarily unavailable (11) Feb 17 18:26:37 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/17) Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:37 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:37 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:37 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:37 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:37 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:37 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:37 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.394 2 Feb 17 18:26:37 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.395 (null) Feb 17 18:26:37 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=395 Feb 17 18:26:37 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_vrouter']: word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.399 2 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.400 (null) Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=400 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='vip__vrouter_pub']/lrm_rsc_op[@id='vip__vrouter_pub_last_0']: @operation_key=vip__vrouter_pub_start_0, @operation=start, @transition-key=27:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;27:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=43, @rc-code=0, @last-run=1455733597, @last-rc-change=1455733597, @exec-time=1385 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/18, version=0.155.400) Feb 17 18:26:38 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.400 to 0.155.399 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource vip__vrouter after start op complete (interval=0) Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/19) Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation vip__vrouter_start_0: ok (node=node-3.test.domain.local, call=44, rc=0, cib-update=19, confirmed=true) Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-vip__vrouter_start_0:44 [ Device "v_vrouter" does not exist.\nSetup default gateway -- do nothing.\n ] Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'vip__vrouter' with start op Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.400 2 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.401 (null) Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=401 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='vip__vrouter']/lrm_rsc_op[@id='vip__vrouter_last_0']: @operation_key=vip__vrouter_start_0, @operation=start, @transition-key=29:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;29:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=44, @rc-code=0, @last-run=1455733597, @last-rc-change=1455733597, @exec-time=1310 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/19, version=0.155.401) Feb 17 18:26:38 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.401 to 0.155.400 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=28:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=vip__vrouter_pub_monitor_5000 Feb 17 18:26:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=45, reply=1, notify=0, exit=4202464 Feb 17 18:26:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:vip__vrouter_pub action:monitor call_id:45 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:38 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=30:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=vip__vrouter_monitor_5000 Feb 17 18:26:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=46, reply=1, notify=0, exit=4202464 Feb 17 18:26:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:vip__vrouter action:monitor call_id:46 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.401 2 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.402 (null) Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=402 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_haproxy']/lrm_rsc_op[@id='p_haproxy_last_0']: @operation_key=p_haproxy_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=60:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;60:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=219, @last-run=1455733597, @last-rc-change=1455733597, @exec-time= Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/268, version=0.155.402) Feb 17 18:26:38 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.402 to 0.155.401 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:38 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:38 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.402 2 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.403 (null) Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=403 Feb 17 18:26:38 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_haproxy']: word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:41 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.404 2 Feb 17 18:26:41 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.405 (null) Feb 17 18:26:41 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=405 Feb 17 18:26:41 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='vip__vrouter_pub']: word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 ocf-ns_IPaddr2: 2016/02/17_18:26:41 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:06:4F:27:DA:C9] 0.789ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.809ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.512ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.542ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_conntrackd_start_0 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=33:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_start_0 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11330 - exited with rc=0 Feb 17 18:26:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11330:stderr [ -- empty -- ] Feb 17 18:26:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11330:stdout [ -- empty -- ] Feb 17 18:26:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:46 pid:11330 exit-code:0 exec-time:3167ms queue-time:0ms Feb 17 18:26:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=47, reply=1, notify=0, exit=4202464 Feb 17 18:26:41 [7495] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_conntrackd action:start call_id:47 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource vip__vrouter after monitor op complete (interval=5000) Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation vip__vrouter_monitor_5000: ok (node=node-3.test.domain.local, call=46, rc=0, cib-update=21, confirmed=false) Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'vip__vrouter' with monitor op Feb 17 18:26:41 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/21) Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:41 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:41 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.405 2 Feb 17 18:26:41 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.406 (null) Feb 17 18:26:41 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=406 Feb 17 18:26:41 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='vip__vrouter']: word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_haproxy_start_0 Feb 17 18:26:41 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=62:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_haproxy_start_0 Feb 17 18:26:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=48, reply=1, notify=0, exit=4202464 Feb 17 18:26:41 [7495] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_haproxy action:start call_id:48 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header ocf-ns_haproxy: 2016/02/17_18:26:41 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:26:41 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:26:41 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_conntrackd: 2016/02/17_18:26:41 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 22:ad:5e:dd:d6:2a brd ff:ff:ff:ff:ff:ff ocf-ns_conntrackd: 2016/02/17_18:26:41 INFO: Starting conntrackd ocf-ns_haproxy: 2016/02/17_18:26:41 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:26:41 INFO: haproxy daemon is not running ocf-ns_haproxy: 2016/02/17_18:26:41 INFO: stdin: is not a tty ocf-ns_haproxy: 2016/02/17_18:26:41 DEBUG: Creating host interface: hapr-host and namespace interface: hapr-ns ocf-ns_haproxy: 2016/02/17_18:26:41 DEBUG: Moving interface: hapr-ns to namespace: haproxy ocf-ns_conntrackd: 2016/02/17_18:26:41 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 22:ad:5e:dd:d6:2a brd ff:ff:ff:ff:ff:ff ocf-ns_haproxy: 2016/02/17_18:26:41 DEBUG: Bringing up host interface: hapr-host ocf-ns_haproxy: 2016/02/17_18:26:41 DEBUG: Setting host interface: hapr-host IP to: 240.0.0.1/30 ocf-ns_haproxy: 2016/02/17_18:26:41 DEBUG: Bringing up the namespace interface: hapr-ns Feb 17 18:26:41 [11829] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:26:41 [11829] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-11829-23) Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [11829] Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [11829] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [11829] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [11829] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0, cpd=0x7fd9d2960884 Feb 17 18:26:41 [5459] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7fd9d29602f0 Feb 17 18:26:42 [11829] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [11829] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5599-11829-23-header Feb 17 18:26:42 [11829] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5599-11829-23-header Feb 17 18:26:42 [11829] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5599-11829-23-header Feb 17 18:26:42 [11829] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-11829-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-11829-23) state:2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5599-11829-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5599-11829-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5599-11829-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-11829-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [11829] Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [11829] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [11829] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [11829] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:42 [11829] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [11829] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-11829-23-header Feb 17 18:26:42 [11829] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-11829-23-header Feb 17 18:26:42 [11829] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-11829-23-header Feb 17 18:26:42 [11829] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:42 [11829] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:26:42 [11829] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-11829-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-11829-23) state:2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-11829-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-11829-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-11829-23-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0xb39e60 for uid=0 gid=0 pid=11862 id=9e4ad2ff-79b6-4ddb-8758-77651f76ec1d Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (7461-11862-14) Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [11862] Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x1c45b80 for uid=0 gid=0 pid=11862 id=f60938a4-a5b1-477f-8d27-3e99c55bd045 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (7543-11862-11) Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11862] Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: master-p_conntrackd=100 Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: master-p_conntrackd=100 for node-3.test.domain.local Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: info: main: Update master-p_conntrackd=100 sent via attrd Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 100, Current: (null), Stored: (null) Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of master-p_conntrackd is 100 Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_conntrackd (100) Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-7461-11862-14-header Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-7461-11862-14-header Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-7461-11862-14-header Feb 17 18:26:42 [11862] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_conntrackd'] does not exist Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (7461-11862-14) Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7461-11862-14) state:2 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-7461-11862-14-header Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 29: master-p_conntrackd=100 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (7543-11862-11) Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7543-11862-11) state:2 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-7461-11862-14-header Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-7543-11862-11-header Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-7543-11862-11-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-7461-11862-14-header Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-7543-11862-11-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/29) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2958840 Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.406 2 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.407 (null) Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=407 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/29, version=0.155.407) Feb 17 18:26:42 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.407 to 0.155.406 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-11882-28) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [11882] Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7fd9d295b010, cpd=0x7fd9d295b714 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header ocf-ns_haproxy: 2016/02/17_18:26:42 DEBUG: Setting namespace interface: hapr-ns IP to: 240.0.0.2/30 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7fd9d295b010 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 29 for master-p_conntrackd=100 passed Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2958840 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5599-11882-28-header Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5599-11882-28-header Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5599-11882-28-header Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-11882-28) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-11882-28) state:2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7fd9d295b010 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5599-11882-28-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5599-11882-28-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5599-11882-28-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-11882-27) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [11882] Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2958840 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/615, version=0.155.407) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-11882-27-header Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-11882-27-header Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-11882-27-header Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:26:42 [11882] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-11882-27) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-11882-27) state:2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2958840 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-11882-27-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-11882-27-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-11882-27-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0xb39e60 for uid=0 gid=0 pid=11920 id=1a6115fa-6ae9-494b-853c-f5daaad0cef9 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (7461-11920-14) Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [11920] Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x1c45b80 for uid=0 gid=0 pid=11920 id=a9f14ae4-d66a-44c3-bb01-38b7bd7076bb Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (7543-11920-11) Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11920] Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: master-p_conntrackd=100 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100 Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: master-p_conntrackd=100 for node-3.test.domain.local Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: info: main: Update master-p_conntrackd=100 sent via attrd Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (7461-11920-14) Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7461-11920-14) state:2 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-7461-11920-14-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-7461-11920-14-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-7461-11920-14-header Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-7461-11920-14-header Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-7461-11920-14-header Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-7461-11920-14-header Feb 17 18:26:42 [11920] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (7543-11920-11) Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7543-11920-11) state:2 Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-7543-11920-11-header Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-7543-11920-11-header Feb 17 18:26:42 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-7543-11920-11-header Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:11533 - exited with rc=0 Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:11533:stderr [ -- empty -- ] Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:11533:stdout [ vrouter ] Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:11533:stdout [ mgmt-conntrd ] Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:11533:stdout [ vrouter ] Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:11533:stdout [ mgmt-conntrd ] Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_conntrackd action:start call_id:47 pid:11533 exit-code:0 exec-time:495ms queue-time:0ms Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11548) = 0: Resource temporarily unavailable (11) Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_conntrackd after start op complete (interval=0) Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_conntrackd_meta-data_0 process 12053 exited with rc=0 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_start_0: ok (node=node-3.test.domain.local, call=47, rc=0, cib-update=22, confirmed=true) Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_conntrackd_start_0:47 [ vrouter\n mgmt-conntrd\nvrouter\n mgmt-conntrd\n ] Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_conntrackd' with start op Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/22) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.407 2 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.408 (null) Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=408 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_conntrackd']/lrm_rsc_op[@id='p_conntrackd_last_0']: @operation_key=p_conntrackd_start_0, @operation=start, @transition-key=33:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;33:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=47, @rc-code=0, @last-run=1455733601, @last-rc-change=1455733601, @exec-time=495 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/22, version=0.155.408) Feb 17 18:26:42 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.408 to 0.155.407 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] ocf-ns_haproxy: 2016/02/17_18:26:42 DEBUG: Creating default route inside the namespace to 240.0.0.1 with metric 10000 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=199:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_notify_0 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=49, reply=1, notify=0, exit=4202464 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_conntrackd action:notify call_id:49 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11548) = 0: Resource temporarily unavailable (11) Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:12100 - exited with rc=0 Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:12100:stderr [ -- empty -- ] Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:12100:stdout [ -- empty -- ] Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_conntrackd action:notify call_id:49 pid:12100 exit-code:0 exec-time:15ms queue-time:0ms Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_conntrackd after notify op complete (interval=0) Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_conntrackd_notify_0 from 199:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455733602-4 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_notify_0: ok (node=node-3.test.domain.local, call=49, rc=0, cib-update=0, confirmed=true) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header ocf-ns_haproxy: 2016/02/17_18:26:42 INFO: net.ipv4.conf.hapr-host.rp_filter = 2 ocf-ns_haproxy: 2016/02/17_18:26:42 INFO: net.ipv4.conf.all.rp_filter = 2 ocf-ns_haproxy: 2016/02/17_18:26:42 DEBUG: Adding route on the host system to 10.109.0.0/24: 240.0.0.2 ocf-ns_haproxy: 2016/02/17_18:26:42 DEBUG: Adding route on the host system to 192.168.2.0/24: 240.0.0.2 ocf-ns_haproxy: 2016/02/17_18:26:42 DEBUG: Adding route on the host system to 192.168.1.0/24: 240.0.0.2 ocf-ns_haproxy: 2016/02/17_18:26:42 DEBUG: Adding route on the host system to 192.168.0.0/24: 240.0.0.2 ocf-ns_haproxy: 2016/02/17_18:26:42 DEBUG: Adding route on the host system to 10.109.1.0/24: 240.0.0.2 ocf-ns_haproxy: 2016/02/17_18:26:42 INFO: Started haproxy daemon. Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_start_0:11548 - exited with rc=0 Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_start_0:11548:stderr [ -- empty -- ] Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_start_0:11548:stdout [ -- empty -- ] Feb 17 18:26:42 [7495] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_haproxy action:start call_id:48 pid:11548 exit-code:0 exec-time:828ms queue-time:0ms Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_haproxy after start op complete (interval=0) Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_haproxy_meta-data_0 process 12171 exited with rc=0 Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: get_rsc_restart_list: Attr dummy is not reloadable Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_haproxy_start_0: ok (node=node-3.test.domain.local, call=48, rc=0, cib-update=23, confirmed=true) Feb 17 18:26:42 [7621] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_haproxy' with start op Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:42 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.408 2 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.409 (null) Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=409 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_haproxy']/lrm_rsc_op[@id='p_haproxy_last_0']: @operation_key=p_haproxy_start_0, @operation=start, @transition-key=62:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;62:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=48, @rc-code=0, @last-run=1455733601, @last-rc-change=1455733601, @exec-time=828 Feb 17 18:26:42 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/23, version=0.155.409) Feb 17 18:26:42 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.409 to 0.155.408 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:42 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:46 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:26:46 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: throttle_cib_load: Init 7 + 10 ticks at 1455733607 (100 tps) Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 3.560000 (full: 3.56 1.15 0.41 1/461 12354) Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: notice: throttle_handle_load: High CPU load detected: 3.560000 Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: throttle_timer_cb: New throttle mode: 0100 (was ffffffff) Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: info: throttle_send_command: Updated throttle state to 0100 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:47 [7621] node-3.test.domain.local crmd: debug: throttle_update: Host node-3.test.domain.local supports a maximum of 4 jobs and throttle mode 0100. New job limit is 1 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:47 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:47 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:47 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:47 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:47 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:47 [7461] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 91598446f7f07d8a36b3f57558955e34 for 0.155.409 (0xb28ff0 0) Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:47 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:47 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:47 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:47 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:47 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header ocf-ns_IPaddr2: 2016/02/17_18:26:49 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.871ms Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.553ms Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.536ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:26:49 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:12317 - exited with rc=0 Feb 17 18:26:49 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:12317:stderr [ -- empty -- ] Feb 17 18:26:49 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:12317:stdout [ -- empty -- ] Feb 17 18:26:49 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:45 pid:12317 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:26:49 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12330) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:26:49 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:06:4F:27:DA:C9] 0.795ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.819ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.546ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.543ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:26:49 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:12330 - exited with rc=0 Feb 17 18:26:49 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:12330:stderr [ -- empty -- ] Feb 17 18:26:49 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:12330:stdout [ -- empty -- ] Feb 17 18:26:49 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:46 pid:12330 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:26:54 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:26:54 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bd70 Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:55 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.409 2 Feb 17 18:26:55 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.410 (null) Feb 17 18:26:55 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=410 Feb 17 18:26:55 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_mysql']/lrm_rsc_op[@id='p_mysql_last_0']: @operation_key=p_mysql_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=68:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;68:69:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=220, @last-run=1455733597, @last-rc-change=1455733597, @exec-time=18353 Feb 17 18:26:55 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/271, version=0.155.410) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bd70 Feb 17 18:26:55 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.410 to 0.155.409 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=71:70:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_haproxy_monitor_30000 Feb 17 18:26:55 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=50, reply=1, notify=0, exit=4202464 Feb 17 18:26:55 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_haproxy action:monitor call_id:50 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header ocf-ns_haproxy: 2016/02/17_18:26:55 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:26:55 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:26:55 INFO: haproxy daemon running Feb 17 18:26:55 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12492) = 0: Resource temporarily unavailable (11) Feb 17 18:26:55 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12505) = 0: Resource temporarily unavailable (11) Feb 17 18:26:55 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:12534 - exited with rc=0 Feb 17 18:26:55 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:12534:stderr [ -- empty -- ] Feb 17 18:26:55 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:12534:stdout [ -- empty -- ] Feb 17 18:26:55 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:50 pid:12534 exit-code:0 exec-time:59ms queue-time:0ms Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_haproxy after monitor op complete (interval=30000) Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_haproxy_monitor_30000: ok (node=node-3.test.domain.local, call=50, rc=0, cib-update=24, confirmed=false) Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_haproxy' with monitor op Feb 17 18:26:55 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/24) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:55 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:55 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.410 2 Feb 17 18:26:55 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.411 (null) Feb 17 18:26:55 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=411 Feb 17 18:26:55 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_haproxy']: word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_conntrackd_promote_0 Feb 17 18:26:55 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=41:70:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_promote_0 Feb 17 18:26:55 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=52, reply=1, notify=0, exit=4202464 Feb 17 18:26:55 [7495] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_conntrackd action:promote call_id:52 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:55 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-12583-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [12583] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0, cpd=0x7fd9d2960884 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7fd9d29602f0 Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5599-12583-23-header Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5599-12583-23-header Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5599-12583-23-header Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-12583-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-12583-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5599-12583-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5599-12583-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5599-12583-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-12583-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [12583] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-12583-23-header Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-12583-23-header Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-12583-23-header Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:26:56 [12583] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-12583-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-12583-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-12583-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-12583-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-12583-23-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0xb35730 for uid=0 gid=0 pid=12585 id=2406b2c3-2050-4221-baff-572ad987e582 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (7461-12585-14) Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12585] Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x1c45b80 for uid=0 gid=0 pid=12585 id=40cdf740-5b6a-45fb-8618-03286df79098 Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (7543-12585-11) Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12585] Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: master-p_conntrackd=1000 Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1000, Current: 100, Stored: 100 Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of master-p_conntrackd is 1000 Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_conntrackd (1000) Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: master-p_conntrackd=1000 for node-3.test.domain.local Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: info: main: Update master-p_conntrackd=1000 sent via attrd Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-7461-12585-14-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_conntrackd'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[2]) Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-7461-12585-14-header Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-7461-12585-14-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (7461-12585-14) Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7461-12585-14) state:2 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:26:56 [12585] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-7461-12585-14-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-7461-12585-14-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-7461-12585-14-header Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 31: master-p_conntrackd=1000 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/31) Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (7543-12585-11) Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7543-12585-11) state:2 Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-7543-12585-11-header Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-7543-12585-11-header Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-7543-12585-11-header Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12492) = 0: Resource temporarily unavailable (11) Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12505) = 0: Resource temporarily unavailable (11) Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_promote_0:12569 - exited with rc=0 Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_promote_0:12569:stderr [ -- empty -- ] Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_promote_0:12569:stdout [ -- empty -- ] Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_conntrackd action:promote call_id:52 pid:12569 exit-code:0 exec-time:313ms queue-time:0ms Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_conntrackd after promote op complete (interval=0) Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_promote_0: ok (node=node-3.test.domain.local, call=52, rc=0, cib-update=25, confirmed=true) Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_conntrackd' with promote op Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-27) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2958840 Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-23-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.412 2 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.413 (null) Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=413 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-master-p_conntrackd']: @value=1000 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/31, version=0.155.413) Feb 17 18:26:56 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 31 for master-p_conntrackd=1000 passed Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-27) Feb 17 18:26:56 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.413 to 0.155.412 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-27) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2958840 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-27-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-27-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-27-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/617, version=0.155.413) Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/25) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.413 2 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.414 (null) Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=414 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_conntrackd']/lrm_rsc_op[@id='p_conntrackd_last_0']: @operation_key=p_conntrackd_promote_0, @operation=promote, @transition-key=41:70:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;41:70:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=52, @last-run=1455733615, @last-rc-change=1455733615, @exec-time=313 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/25, version=0.155.414) Feb 17 18:26:56 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.414 to 0.155.413 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=216:70:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_notify_0 Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=53, reply=1, notify=0, exit=4202464 Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_conntrackd action:notify call_id:53 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12492) = 0: Resource temporarily unavailable (11) Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12505) = 0: Resource temporarily unavailable (11) Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:12586 - exited with rc=0 Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:12586:stderr [ -- empty -- ] Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:12586:stdout [ -- empty -- ] Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_conntrackd action:notify call_id:53 pid:12586 exit-code:0 exec-time:36ms queue-time:0ms Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_conntrackd after notify op complete (interval=0) Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_conntrackd_notify_0 from 216:70:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455733616-7 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_notify_0: ok (node=node-3.test.domain.local, call=53, rc=0, cib-update=0, confirmed=true) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.414 2 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.415 (null) Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=415 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_conntrackd']: word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=45:71:8:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_monitor_27000 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=54, reply=1, notify=0, exit=4202464 Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_conntrackd action:monitor call_id:54 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header ocf-ns_conntrackd: 2016/02/17_18:26:56 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 22:ad:5e:dd:d6:2a brd ff:ff:ff:ff:ff:ff Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12492) = 0: Resource temporarily unavailable (11) Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12505) = 0: Resource temporarily unavailable (11) Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:12595 - exited with rc=8 Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:12595:stderr [ -- empty -- ] Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:12595:stdout [ vrouter ] Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:12595:stdout [ mgmt-conntrd ] Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:54 pid:12595 exit-code:8 exec-time:140ms queue-time:0ms Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_conntrackd after monitor op complete (interval=27000) Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_monitor_27000: master (node=node-3.test.domain.local, call=54, rc=8, cib-update=26, confirmed=false) Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: node-3.test.domain.local-p_conntrackd_monitor_27000:54 [ vrouter\n mgmt-conntrd\n ] Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_conntrackd' with monitor op Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/26) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:56 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.415 2 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.416 (null) Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=416 Feb 17 18:26:56 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_conntrackd']: word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_rabbitmq-server' not found (9 active resources) Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_rabbitmq-server:1' not found (9 active resources) Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_rabbitmq-server' to the rsc list (10 active resources) Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:56 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=13:71:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_rabbitmq-server_monitor_0 Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=59, reply=1, notify=0, exit=4202464 Feb 17 18:26:56 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_rabbitmq-server action:monitor call_id:59 Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-12632-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [12632] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0, cpd=0x7fd9d2960884 Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7fd9d29602f0 Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5599-12632-23-header Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5599-12632-23-header Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5599-12632-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-12632-23) Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-12632-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5599-12632-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5599-12632-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5599-12632-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-12632-23) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [12632] Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-12632-23-header Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-12632-23-header Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-12632-23-header Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-12632-23) Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:26:56 [12632] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-12632-23) state:2 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-12632-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-12632-23-header Feb 17 18:26:56 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-12632-23-header lrmd: 2016/02/17_18:26:56 DEBUG: p_rabbitmq-server: monitor: action start. lrmd: 2016/02/17_18:26:56 INFO: p_rabbitmq-server: get_monitor(): CHECK LEVEL IS: 0 lrmd: 2016/02/17_18:26:57 INFO: get_status() failed with code 139. Command output: Error: unable to connect to node 'rabbit@messaging-node-3': nodedown DIAGNOSTICS =========== attempted to contact: ['rabbit@messaging-node-3'] rabbit@messaging-node-3: * connected to epmd (port 4369) on messaging-node-3 * epmd reports: node 'rabbit' not running at all other nodes on messaging-node-3: ['rabbitmq-cli-12755'] * suggestion: start the node current node details: - node name: 'rabbitmq-cli-12755@node-3' - home dir: /var/lib/rabbitmq - cookie hash: soeIWU2jk2YNseTyDSlsEA== Segmentation fault (core dumped) lrmd: 2016/02/17_18:26:57 INFO: p_rabbitmq-server: get_monitor(): get_status() returns 7. lrmd: 2016/02/17_18:26:57 INFO: p_rabbitmq-server: get_monitor(): ensuring this slave does not get promoted. lrmd: 2016/02/17_18:26:57 INFO: p_rabbitmq-server: master_score(): Updating master score attribute with 0 Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-12850-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [12850] Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0, cpd=0x7fd9d2960884 Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7fd9d29602f0 Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-12850-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-12850-23) state:2 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5599-12850-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5599-12850-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5599-12850-23-header Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5599-12850-23-header Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5599-12850-23-header Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5599-12850-23-header Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-12850-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [12850] Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-12850-23-header Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-12850-23-header Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-12850-23-header Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:26:57 [12850] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-12850-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-12850-23) state:2 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-12850-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-12850-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-12850-23-header Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0xbf9e30 for uid=0 gid=0 pid=12852 id=2cd3edc0-369e-4ea8-8670-058b5670246e Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (7461-12852-14) Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12852] Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x1c69b50 for uid=0 gid=0 pid=12852 id=06239efd-4c09-447a-96e2-09ec6c090c27 Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (7543-12852-11) Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12852] Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: master-p_rabbitmq-server=0 Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: (null), Stored: (null) Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of master-p_rabbitmq-server is 0 Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_rabbitmq-server (0) Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: master-p_rabbitmq-server=0 for node-3.test.domain.local Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: info: main: Update master-p_rabbitmq-server=0 sent via attrd Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-7461-12852-14-header Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-7461-12852-14-header Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-7461-12852-14-header Feb 17 18:26:57 [12852] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_rabbitmq-server'] does not exist Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (7461-12852-14) Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7461-12852-14) state:2 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-7461-12852-14-header Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 33: master-p_rabbitmq-server=0 Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (7543-12852-11) Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7543-12852-11) state:2 Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-7461-12852-14-header Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-7543-12852-11-header Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-7543-12852-11-header Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-7543-12852-11-header Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-7461-12852-14-header Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/33) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5be20 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2958840 Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.416 2 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.417 (null) Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=417 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/33, version=0.155.417) Feb 17 18:26:57 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.417 to 0.155.416 lrmd: 2016/02/17_18:26:57 DEBUG: p_rabbitmq-server: monitor: role: Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5be20 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:57 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 33 for master-p_rabbitmq-server=0 passed Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 lrmd: 2016/02/17_18:26:57 DEBUG: p_rabbitmq-server: monitor: result: 7 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5be20 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2958840 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header lrmd: 2016/02/17_18:26:57 DEBUG: p_rabbitmq-server: monitor: action end. Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/619, version=0.155.417) Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12492) = 0: Resource temporarily unavailable (11) Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12505) = 0: Resource temporarily unavailable (11) Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_monitor_0:12616 - exited with rc=7 Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:12616:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:12616:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:12616:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:12616:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:12616:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:12616:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_monitor_0:12616:stdout [ -- empty -- ] Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_rabbitmq-server action:monitor call_id:59 pid:12616 exit-code:7 exec-time:682ms queue-time:0ms Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_rabbitmq-server after monitor op complete (interval=0) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5be20 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-12882-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [12882] Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0, cpd=0x7fd9d2960884 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7fd9d29602f0 Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5599-12882-23-header Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5599-12882-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-12882-23) Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5599-12882-23-header Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-12882-23) state:2 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5599-12882-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5599-12882-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5599-12882-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-12882-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [12882] Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-12882-23-header Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-12882-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-12882-23) Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-12882-23-header Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-12882-23) state:2 Feb 17 18:26:57 [12882] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-12882-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-12882-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-12882-23-header Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: info: services_os_action_execute: Managed rabbitmq-server_meta-data_0 process 12866 exited with rc=0 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/27) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bdf0 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_rabbitmq-server_monitor_0: not running (node=node-3.test.domain.local, call=59, rc=7, cib-update=27, confirmed=true) Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: node-3.test.domain.local-p_rabbitmq-server_monitor_0:59 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n ] Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_rabbitmq-server' with monitor op Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.417 2 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.418 (null) Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=418 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/27, version=0.155.418) Feb 17 18:26:57 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.418 to 0.155.417 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bdf0 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bdf0 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bdf0 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_dns' not found (10 active resources) Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_dns:1' not found (10 active resources) Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_dns' to the rsc list (11 active resources) Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=15:72:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_dns_monitor_0 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=64, reply=1, notify=0, exit=4202464 Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_dns action:monitor call_id:64 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header ocf-ns_dns: 2016/02/17_18:26:57 INFO: dnsmasq daemon is not running Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12492) = 0: Resource temporarily unavailable (11) Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12505) = 0: Resource temporarily unavailable (11) Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_0:12894 - exited with rc=7 Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_0:12894:stderr [ -- empty -- ] Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_0:12894:stdout [ -- empty -- ] Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:64 pid:12894 exit-code:7 exec-time:27ms queue-time:0ms Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_dns after monitor op complete (interval=0) Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_dns_meta-data_0 process 12904 exited with rc=0 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/28) Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_dns_monitor_0: not running (node=node-3.test.domain.local, call=64, rc=7, cib-update=28, confirmed=true) Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_dns' with monitor op Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:57 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.418 2 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.419 (null) Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=419 Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:26:57 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/28, version=0.155.419) Feb 17 18:26:57 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.419 to 0.155.418 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_mysql_start_0 Feb 17 18:26:57 [7621] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=79:72:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_mysql_start_0 Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 5488a65d-580e-45a9-9600-4c95e6cd32de: rc=65, reply=1, notify=0, exit=4202464 Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_mysql action:start call_id:65 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:26:57 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header ocf-ns_IPaddr2: 2016/02/17_18:26:57 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.745ms Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.534ms Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.545ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:12492 - exited with rc=0 Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:12492:stderr [ -- empty -- ] Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:12492:stdout [ -- empty -- ] Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:45 pid:12492 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12505) = 0: Resource temporarily unavailable (11) Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:26:57 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:06:4F:27:DA:C9] 0.627ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.645ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.526ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.547ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:12505 - exited with rc=0 Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:12505:stderr [ -- empty -- ] Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:12505:stdout [ -- empty -- ] Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:46 pid:12505 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:26:57 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Resource temporarily unavailable (11) Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:02 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:02 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:02 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:27:02 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:02 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:27:02 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:27:02 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:27:02 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:27:02 [7461] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: fc894912da537f198e8d23a9b194bdcf for 0.155.419 (0xb28ff0 0) Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:02 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:02 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:02 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5be90 Feb 17 18:27:02 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:02 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:27:02 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:27:02 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5be90 Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:27:02 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header ocf-mysql-wss: 2016/02/17_18:27:02 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:27:02 INFO: MySQL is not running ocf-mysql-wss: 2016/02/17_18:27:02 INFO: Create socket dir: /var/run/mysqld and chown to mysql:mysql ocf-mysql-wss: 2016/02/17_18:27:02 INFO: Create PID dir: /var/run/resource-agents/mysql-wss and chown to mysql:mysql Feb 17 18:27:02 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:27:02 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:27:05 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.795ms Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.535ms Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.531ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:27:05 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Interrupted system call (4) Feb 17 18:27:05 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:13113 - exited with rc=0 Feb 17 18:27:05 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:13113:stderr [ -- empty -- ] Feb 17 18:27:05 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:13113:stdout [ -- empty -- ] Feb 17 18:27:05 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:45 pid:13113 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:27:05 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13126) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:27:05 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:06:4F:27:DA:C9] 0.724ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.752ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.529ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.543ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:27:05 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Resource temporarily unavailable (11) Feb 17 18:27:05 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:13126 - exited with rc=0 Feb 17 18:27:05 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:13126:stderr [ -- empty -- ] Feb 17 18:27:05 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:13126:stdout [ -- empty -- ] Feb 17 18:27:05 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:46 pid:13126 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:27:07 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:27:07 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:27:07 INFO: net.ipv4.ip_forward = 1 Feb 17 18:27:07 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Resource temporarily unavailable (11) Feb 17 18:27:07 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:13336 - exited with rc=0 Feb 17 18:27:07 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:13336:stderr [ -- empty -- ] Feb 17 18:27:07 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:13336:stdout [ -- empty -- ] Feb 17 18:27:07 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:42 pid:13336 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:27:07 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:27:07 ERROR: MySQL is not running ocf-mysql-wss: 2016/02/17_18:27:07 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:20774 ocf-mysql-wss: 2016/02/17_18:27:07 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:20774 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0xcc1cc0 for uid=0 gid=0 pid=13493 id=ec9cccb7-92af-4c8d-81ab-4a88766ffc72 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (7461-13493-14) Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13493] Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x1c69b50 for uid=0 gid=0 pid=13493 id=ea04cb4b-9bca-4c20-b498-91604f716257 Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (7543-13493-11) Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13493] Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:20774 Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 71dae69f-d579-11e5-a980-12eca8fb60f4:20774, Current: (null), Stored: (null) Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of gtid is 71dae69f-d579-11e5-a980-12eca8fb60f4:20774 Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:20774 for node-3.test.domain.local Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: info: main: Update gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:20774 sent via attrd Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-7461-13493-14-header Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-7461-13493-14-header Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-7461-13493-14-header Feb 17 18:27:07 [13493] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (7461-13493-14) Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7461-13493-14) state:2 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-7461-13493-14-header Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-7461-13493-14-header Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-7461-13493-14-header Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: gtid (71dae69f-d579-11e5-a980-12eca8fb60f4:20774) Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] does not exist Feb 17 18:27:07 [7461] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/35) Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 35: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:20774 Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (7543-13493-11) Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7543-13493-11) state:2 Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-7543-13493-11-header Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-7543-13493-11-header Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-7543-13493-11-header Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [13495] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:27:07 [13495] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:27:07 [13495] node-3.test.domain.local crm_node: debug: main: Attempting to process -q command for cluster type: corosync Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5be90 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-13495-27) Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [13495] Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7fd9d2958840 Feb 17 18:27:07 [13495] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [13495] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [13495] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7fd9d2958840 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:27:07 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.419 2 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.420 (null) Feb 17 18:27:07 [7461] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=420 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:27:07 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/35, version=0.155.420) Feb 17 18:27:07 [7473] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.420 to 0.155.419 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5be90 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7fd9d2958840 Feb 17 18:27:07 [13495] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:07 [13495] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-request-5599-13495-27-header Feb 17 18:27:07 [13495] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-response-5599-13495-27-header Feb 17 18:27:07 [13495] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-event-5599-13495-27-header Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-13495-27) Feb 17 18:27:07 [13495] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-13495-27) state:2 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7fd9d2958840 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-5599-13495-27-header Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-5599-13495-27-header Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-5599-13495-27-header Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 35 for gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:20774 passed Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d29602f0 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7543-27) Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7543] Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2958840 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:27:07 [7461] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/621, version=0.155.420) Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d29602f0 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:07 [13504] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:27:07 [13504] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:27:07 [13504] node-3.test.domain.local crm_node: debug: main: Attempting to process -l command for cluster type: corosync Feb 17 18:27:07 [7299] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x14b0900 for uid=0 gid=0 pid=13504 id=931eb585-9de5-4aa8-a90f-0023b57f0110 Feb 17 18:27:07 [7299] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (7299-13504-12) Feb 17 18:27:07 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [13504] Feb 17 18:27:07 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:27:07 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:27:07 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:27:07 [13504] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:27:07 [13504] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:27:07 [13504] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:27:07 [13504] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:27:07 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (7299-13504-12) Feb 17 18:27:07 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7299-13504-12) state:2 Feb 17 18:27:07 [7299] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:27:07 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-7299-13504-12-header Feb 17 18:27:07 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-7299-13504-12-header Feb 17 18:27:07 [7299] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-7299-13504-12-header Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:27:07 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7543-27) Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-27) state:2 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2958840 Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7543-27-header Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7543-27-header Feb 17 18:27:07 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7543-27-header Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0xcc1cc0 for uid=0 gid=0 pid=13508 id=6230bf16-33bf-4a33-af13-496f48a5d68b Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (7461-13508-14) Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13508] Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: unpack_config: Stop all active resources: false Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: unpack_config: Default stickiness: 0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:1 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:0 Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (7461-13508-14) Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7461-13508-14) state:2 Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-7461-13508-14-header Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-7461-13508-14-header Feb 17 18:27:07 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-7461-13508-14-header Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-7461-13508-14-header Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-7461-13508-14-header Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-7461-13508-14-header Feb 17 18:27:07 [13508] node-3.test.domain.local crm_resource: info: crm_xml_cleanup: Cleaning up memory from libxml2 ocf-mysql-wss: 2016/02/17_18:27:07 INFO: Starting MySQL Feb 17 18:27:10 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:27:10 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:12 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:12 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:12 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d295a960 Feb 17 18:27:12 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:12 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:27:12 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:27:12 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:27:12 [7461] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:27:12 [7461] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: ee64c95ce7ce41485f30440f373bc60b for 0.155.420 (0xb28ff0 0) Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d295a960 Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7461-23) Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7461] Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:12 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:12 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:12 [7461] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bcb0 Feb 17 18:27:12 [7461] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:12 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header Feb 17 18:27:12 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:27:12 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7461-23) Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-23) state:2 Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bcb0 Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7461-23-header Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7461-23-header Feb 17 18:27:12 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7461-23-header ocf-mysql-wss: 2016/02/17_18:27:12 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:27:12 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:27:13 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.818ms Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.528ms Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.537ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:27:13 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Interrupted system call (4) Feb 17 18:27:13 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14323 - exited with rc=0 Feb 17 18:27:13 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14323:stderr [ -- empty -- ] Feb 17 18:27:13 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14323:stdout [ -- empty -- ] Feb 17 18:27:13 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:45 pid:14323 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:27:13 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14336) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:27:13 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:06:4F:27:DA:C9] 0.692ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.715ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.533ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.533ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:27:13 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Resource temporarily unavailable (11) Feb 17 18:27:13 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14336 - exited with rc=0 Feb 17 18:27:13 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14336:stderr [ -- empty -- ] Feb 17 18:27:13 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14336:stdout [ -- empty -- ] Feb 17 18:27:13 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:46 pid:14336 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.001333 (4 ticks in 30s) Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 2.340000 (full: 2.34 1.08 0.41 2/448 14438) Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: throttle_handle_load: Noticable CPU load detected: 2.340000 Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: throttle_timer_cb: New throttle mode: 0001 (was 0100) Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: info: throttle_send_command: Updated throttle state to 0001 Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb50 Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb50 Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5599-7621-23) Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [7621] Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7fd9d2c5bb50 Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:27:17 [7621] node-3.test.domain.local crmd: debug: throttle_update: Host node-3.test.domain.local supports a maximum of 4 jobs and throttle mode 0001. New job limit is 2 Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5599-7621-23) Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-23) state:2 Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7fd9d2c5bb50 Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5599-7621-23-header Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5599-7621-23-header Feb 17 18:27:17 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5599-7621-23-header Feb 17 18:27:18 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:27:18 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 ocf-mysql-wss: 2016/02/17_18:27:20 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:27:20 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:27:21 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.850ms Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.542ms Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.533ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:27:21 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Interrupted system call (4) Feb 17 18:27:21 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14448 - exited with rc=0 Feb 17 18:27:21 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14448:stderr [ -- empty -- ] Feb 17 18:27:21 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14448:stdout [ -- empty -- ] Feb 17 18:27:21 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:45 pid:14448 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:27:21 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14467) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:27:22 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:06:4F:27:DA:C9] 0.663ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.698ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.527ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.544ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:27:22 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Resource temporarily unavailable (11) Feb 17 18:27:22 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14467 - exited with rc=0 Feb 17 18:27:22 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14467:stderr [ -- empty -- ] Feb 17 18:27:22 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14467:stdout [ -- empty -- ] Feb 17 18:27:22 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:46 pid:14467 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:27:23 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:27:23 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 22:ad:5e:dd:d6:2a brd ff:ff:ff:ff:ff:ff Feb 17 18:27:23 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Resource temporarily unavailable (11) Feb 17 18:27:23 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:14560 - exited with rc=8 Feb 17 18:27:23 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:14560:stderr [ -- empty -- ] Feb 17 18:27:23 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:14560:stdout [ vrouter ] Feb 17 18:27:23 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:14560:stdout [ mgmt-conntrd ] Feb 17 18:27:23 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:54 pid:14560 exit-code:8 exec-time:0ms queue-time:0ms Feb 17 18:27:25 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:27:25 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:27:25 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:27:25 INFO: haproxy daemon running Feb 17 18:27:25 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Resource temporarily unavailable (11) Feb 17 18:27:25 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:14592 - exited with rc=0 Feb 17 18:27:25 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:14592:stderr [ -- empty -- ] Feb 17 18:27:25 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:14592:stdout [ -- empty -- ] Feb 17 18:27:25 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:50 pid:14592 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:27:26 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:27:27 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 ocf-mysql-wss: 2016/02/17_18:27:28 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:27:28 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:27:30 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.876ms Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.547ms Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.545ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:27:30 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Interrupted system call (4) Feb 17 18:27:30 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14627 - exited with rc=0 Feb 17 18:27:30 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14627:stderr [ -- empty -- ] Feb 17 18:27:30 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14627:stdout [ -- empty -- ] Feb 17 18:27:30 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:45 pid:14627 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:27:30 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14646) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:27:30 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:06:4F:27:DA:C9] 0.621ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.654ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.533ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.542ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:27:30 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Resource temporarily unavailable (11) Feb 17 18:27:30 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14646 - exited with rc=0 Feb 17 18:27:30 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14646:stderr [ -- empty -- ] Feb 17 18:27:30 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14646:stdout [ -- empty -- ] Feb 17 18:27:30 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:46 pid:14646 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:27:35 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:27:35 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 ocf-mysql-wss: 2016/02/17_18:27:36 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:27:36 INFO: MySQL is not running Feb 17 18:27:37 [7495] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:27:37 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:27:37 INFO: net.ipv4.ip_forward = 1 Feb 17 18:27:37 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Resource temporarily unavailable (11) Feb 17 18:27:37 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14746) = 0: Resource temporarily unavailable (11) Feb 17 18:27:37 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14765) = 0: Resource temporarily unavailable (11) Feb 17 18:27:37 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:14802 - exited with rc=0 Feb 17 18:27:37 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:14802:stderr [ -- empty -- ] Feb 17 18:27:37 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:14802:stdout [ -- empty -- ] Feb 17 18:27:37 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:42 pid:14802 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:27:38 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.734ms Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.528ms Unicast reply from 10.109.1.2 [BA:EE:D5:CC:A0:41] 0.541ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:27:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Interrupted system call (4) Feb 17 18:27:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14746 - exited with rc=0 Feb 17 18:27:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14746:stderr [ -- empty -- ] Feb 17 18:27:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14746:stdout [ -- empty -- ] Feb 17 18:27:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:45 pid:14746 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:27:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14765) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:27:38 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:06:4F:27:DA:C9] 0.801ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.827ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.525ms Unicast reply from 192.168.0.1 [22:AD:5E:DD:D6:2A] 0.544ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:27:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12908) = 0: Resource temporarily unavailable (11) Feb 17 18:27:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14765 - exited with rc=0 Feb 17 18:27:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14765:stderr [ -- empty -- ] Feb 17 18:27:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14765:stdout [ -- empty -- ] Feb 17 18:27:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:46 pid:14765 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:27:40 [5459] node-3.test.domain.local corosync notice [MAIN ] sig_exit_handler Node was shut down by a signal Feb 17 18:27:40 [5459] node-3.test.domain.local corosync notice [SERV ] service_exit_schedwrk_handler Unloading all Corosync service engines. Feb 17 18:27:40 [5459] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_withdraw withdrawing server sockets Feb 17 18:27:40 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_unref qb_ipcs_unref() - destroying Feb 17 18:27:40 [5459] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync vote quorum service v1.0 Feb 17 18:27:40 [5459] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_withdraw withdrawing server sockets Feb 17 18:27:40 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_unref qb_ipcs_unref() - destroying Feb 17 18:27:40 [5459] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync configuration map access Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-18) state:2 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipc_us_ready: poll(fd 6) got POLLHUP Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cfg-response-5599-7299-18-header Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: error: cfg_connection_destroy: Connection destroyed Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: notice: pcmk_shutdown_worker: Shuting down Pacemaker Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cfg-event-5599-7299-18-header Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: notice: stop_child: Stopping crmd: Sent -15 to process 7621 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cfg-request-5599-7299-18-header Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: info: crm_signal_dispatch: Invoking handler for signal 15: Terminated Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: notice: crm_shutdown: Requesting shutdown, upper limit is 1200000ms Feb 17 18:27:41 [5459] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_withdraw withdrawing server sockets Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: crm_timer_start: Started Shutdown Escalation (I_STOP:1200000ms), src=44 Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_SHUTDOWN: [ state=S_NOT_DC cause=C_SHUTDOWN origin=crm_shutdown ] Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_unref qb_ipcs_unref() - destroying Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: info: do_shutdown_req: Sending shutdown request to node-2.test.domain.local Feb 17 18:27:41 [5459] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync configuration service Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-21) state:2 Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: _check_connection_state: interpreting result -107 as a disconnect: Transport endpoint is not connected (107) Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: info: crm_cs_flush: Sent 0 CPG messages (1 remaining, last=9): Library error (2) Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: qb_ipc_us_ready: poll(fd 7) got POLLHUP Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7fd9d2957c00 Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: error: pcmk_cpg_dispatch: Connection to the CPG API failed: Library error (2) Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: error: crmd_cs_destroy: connection terminated Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-attrd-request-7543-7621-9-header Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-attrd-response-7543-7621-9-header Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-attrd-event-7543-7621-9-header Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: stonith_api_signoff: Signing out of the STONITH Service Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-stonith-ng-request-7473-7621-10-header Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-stonith-ng-response-7473-7621-10-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5599-7621-21-header Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-stonith-ng-event-7473-7621-10-header Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: debug: get_xpath_object: No match for //st_notify_disconnect in /notify Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: info: tengine_stonith_connection_destroy: Fencing daemon disconnected Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5599-7621-21-header Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: notice: crmd_exit: Forcing immediate exit: Link has been severed (67) Feb 17 18:27:41 [7621] node-3.test.domain.local crmd: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5599-7621-21-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7543-26) state:2 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7fd9d295e6a0 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5599-7543-26-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5599-7543-26-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5599-7543-26-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7461-24) state:2 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7fd9d2959950 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5599-7461-24-header Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: debug: qb_ipc_us_ready: poll(fd 6) got POLLHUP Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: error: pcmk_cpg_dispatch: Connection to the CPG API failed: Library error (2) Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: crit: attrd_cs_destroy: Lost connection to Corosync service! Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: notice: main: Exiting... Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: notice: main: Disconnecting client 0x1b9dd70, pid=7621... Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7543-7621-9) state:2 Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-7543-7621-9-header Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-7543-7621-9-header Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-7543-7621-9-header Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-7461-7543-13-header Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-7461-7543-13-header Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-7461-7543-13-header Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(7461) = 0: Resource temporarily unavailable (11) Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(7473) = 0: Resource temporarily unavailable (11) Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(7495) = 0: Resource temporarily unavailable (11) Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: error: attrd_cib_connection_destroy: Connection to the CIB terminated... Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(7543) = 0: Resource temporarily unavailable (11) Feb 17 18:27:41 [7543] node-3.test.domain.local attrd: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(7585) = 0: Resource temporarily unavailable (11) Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: error: pcmk_child_exit: Child process crmd (7621) exited: Link has been severed (67) Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_ipcs_dispatch_connection_request: HUP conn (7473-7621-10) Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7473-7621-10) state:2 Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-stonith-ng-response-7473-7621-10-header Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-stonith-ng-event-7473-7621-10-header Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-stonith-ng-request-7473-7621-10-header Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: qb_ipcs_dispatch_connection_request: HUP conn (7495-7621-7) Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7495-7621-7) state:2 Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_vrouter_monitor_30000 Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:42 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: warning: qb_ipcs_event_sendv: new_event_notification (7495-7621-7): Bad file descriptor (9) Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/5488a65d-580e-45a9-9600-4c95e6cd32de failed Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation vip__vrouter_pub_monitor_5000 Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:45 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/5488a65d-580e-45a9-9600-4c95e6cd32de failed Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_conntrackd_monitor_27000 Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:54 exit-code:8 exec-time:0ms queue-time:0ms Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/5488a65d-580e-45a9-9600-4c95e6cd32de failed Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_haproxy_monitor_30000 Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:50 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5599-7461-24-header Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/5488a65d-580e-45a9-9600-4c95e6cd32de failed Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation vip__vrouter_monitor_5000 Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:46 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/5488a65d-580e-45a9-9600-4c95e6cd32de failed Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5599-7461-24-header Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: crm_client_destroy: Destroying 5 events Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-lrmd-response-7495-7621-7-header Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-lrmd-event-7495-7621-7-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7473-22) state:2 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:41 [7495] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-lrmd-request-7495-7621-7-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7fd9d2956500 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_ipc_us_ready: poll(fd 6) got POLLHUP Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5599-7473-22-header Feb 17 18:27:41 [7461] node-3.test.domain.local cib: error: pcmk_cpg_dispatch: Connection to the CPG API failed: Library error (2) Feb 17 18:27:41 [7461] node-3.test.domain.local cib: error: cib_cs_destroy: Corosync connection lost! Exiting. Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: uninitializeCib: Deallocating the CIB. Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5599-7473-22-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5599-7473-22-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-19) state:2 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7fd9d2651c00 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5599-7299-19-header Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: uninitializeCib: The CIB has been deallocated. Feb 17 18:27:41 [7461] node-3.test.domain.local cib: info: terminate_cib: cib_cs_destroy: Exiting fast... Feb 17 18:27:41 [7461] node-3.test.domain.local cib: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5599-7299-19-header Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7461-7543-13) state:2 Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-7461-7543-13-header Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-7461-7543-13-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5599-7299-19-header Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-7461-7543-13-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_withdraw withdrawing server sockets Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_unref qb_ipcs_unref() - destroying Feb 17 18:27:41 [5459] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync cluster closed process group service v1.01 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7621-25) state:2 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7fd9d295dbf0 Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7461-7473-12) state:2 Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-7461-7473-12-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-5599-7621-25-header Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-7461-7473-12-header Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-7461-7473-12-header Feb 17 18:27:41 [7461] node-3.test.domain.local cib: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(7461-7621-11) state:2 Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_shm-response-7461-7621-11-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-5599-7621-25-header Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_shm-event-7461-7621-11-header Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_shm-request-7461-7621-11-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-5599-7621-25-header Feb 17 18:27:41 [7461] node-3.test.domain.local cib: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:27:41 [7461] node-3.test.domain.local cib: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:27:41 [7461] node-3.test.domain.local cib: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5599-7299-20) state:2 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7fd9d2954050 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-5599-7299-20-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-5599-7299-20-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-5599-7299-20-header Feb 17 18:27:41 [5459] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_withdraw withdrawing server sockets Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [QB ] qb_ipcs_unref qb_ipcs_unref() - destroying Feb 17 18:27:41 [5459] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync cluster quorum service v0.1 Feb 17 18:27:41 [5459] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync profile loading service Feb 17 18:27:41 [5459] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync watchdog service Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_ipc_us_ready: poll(fd 8) got POLLHUP Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_ipc_us_ready: poll(fd 8) got POLLHUP Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: error: crm_ipc_read: Connection to cib_rw failed Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: error: mainloop_gio_callback: Connection to cib_rw[0x2251ce0] closed (I/O condition=17) Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_ipc_us_ready: poll(fd 8) got POLLHUP Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-request-7461-7473-12-data: No such file or directory (2) Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-request-7461-7473-12-header: No such file or directory (2) Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-response-7461-7473-12-data: No such file or directory (2) Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-response-7461-7473-12-header: No such file or directory (2) Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-event-7461-7473-12-data: No such file or directory (2) Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-event-7461-7473-12-header: No such file or directory (2) Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: notice: cib_connection_destroy: Connection to the CIB terminated. Shutting down. Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: info: stonith_shutdown: Terminating with 0 clients Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: info: main: Done Feb 17 18:27:41 [7473] node-3.test.domain.local stonithd: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s Feb 17 18:27:41 [5459] node-3.test.domain.local corosync debug [TOTEM ] memb_leave_message_send sending join/leave message Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:41 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s Feb 17 18:27:41 [5459] node-3.test.domain.local corosync notice [MAIN ] _corosync_exit_error Corosync Cluster Engine exiting normally Feb 17 18:27:42 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:42 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s ocf-mysql-wss: 2016/02/17_18:27:44 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:27:44 INFO: MySQL is not running Feb 17 18:27:46 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:46 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s Feb 17 18:27:51 [7299] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:27:51 [7299] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000111110 (was 00000000000000000000000000111310) Feb 17 18:27:51 [7299] node-3.test.domain.local pacemakerd: info: crm_cs_flush: Sent 0 CPG messages (1 remaining, last=7): Library error (2) Feb 17 18:27:51 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:51 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:27:52 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:52 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s ocf-mysql-wss: 2016/02/17_18:27:52 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:27:52 INFO: MySQL is not running Feb 17 18:27:54 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:54 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s Feb 17 18:27:57 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:27:57 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s ocf-mysql-wss: 2016/02/17_18:28:00 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:28:00 INFO: MySQL is not running Feb 17 18:28:01 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:01 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s Feb 17 18:28:06 [7299] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:28:06 [7299] node-3.test.domain.local pacemakerd: info: crm_signal_dispatch: Invoking handler for signal 15: Terminated Feb 17 18:28:06 [7299] node-3.test.domain.local pacemakerd: error: pcmk_child_exit: Child process cib (7461) exited: Invalid argument (22) Feb 17 18:28:06 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:06 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:28:07 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:07 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s ocf-mysql-wss: 2016/02/17_18:28:08 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:28:08 INFO: MySQL is not running Feb 17 18:28:09 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:09 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s Feb 17 18:28:12 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:12 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s Feb 17 18:28:16 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:16 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s ocf-mysql-wss: 2016/02/17_18:28:16 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:28:16 INFO: MySQL is not running Feb 17 18:28:21 [7299] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:28:21 [7299] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000111010 (was 00000000000000000000000000111110) Feb 17 18:28:21 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:21 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:28:22 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:22 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s Feb 17 18:28:24 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:24 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s ocf-mysql-wss: 2016/02/17_18:28:24 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:28:24 INFO: MySQL is not running Feb 17 18:28:27 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:27 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s Feb 17 18:28:31 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:31 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s ocf-mysql-wss: 2016/02/17_18:28:32 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:28:32 INFO: MySQL is not running Feb 17 18:28:36 [7299] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:28:36 [7299] node-3.test.domain.local pacemakerd: info: pcmk_child_exit: Child process stonith-ng (7473) exited: OK (0) Feb 17 18:28:36 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:36 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:28:37 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:37 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s Feb 17 18:28:39 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:39 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s ocf-mysql-wss: 2016/02/17_18:28:40 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:28:40 INFO: MySQL is not running Feb 17 18:28:42 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:42 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s Feb 17 18:28:46 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:46 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s ocf-mysql-wss: 2016/02/17_18:28:48 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:28:48 INFO: MySQL is not running Feb 17 18:28:51 [7299] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:28:51 [7299] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000011010 (was 00000000000000000000000000111010) Feb 17 18:28:51 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:51 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:28:52 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:52 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s Feb 17 18:28:54 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:54 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s ocf-mysql-wss: 2016/02/17_18:28:56 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:28:56 INFO: MySQL is not running Feb 17 18:28:57 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:28:57 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s Feb 17 18:29:01 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:29:01 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s ocf-mysql-wss: 2016/02/17_18:29:04 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:29:04 INFO: MySQL is not running Feb 17 18:29:06 [7299] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:29:06 [7299] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(7495) = 0: Connection refused (111) Feb 17 18:29:06 [7299] node-3.test.domain.local pacemakerd: error: pcmk_child_exit: Child process attrd (7543) exited: Transport endpoint is not connected (107) Feb 17 18:29:06 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:29:06 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:29:07 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:29:07 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s Feb 17 18:29:09 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:29:09 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s Feb 17 18:29:12 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:29:12 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s ocf-mysql-wss: 2016/02/17_18:29:12 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:29:12 INFO: MySQL is not running Feb 17 18:29:16 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:29:16 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s ocf-mysql-wss: 2016/02/17_18:29:20 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:29:20 INFO: MySQL is not running Feb 17 18:29:21 [7299] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:29:21 [7299] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000010010 (was 00000000000000000000000000011010) Feb 17 18:29:21 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:29:21 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:29:22 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:29:22 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s Feb 17 18:29:24 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:29:24 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s Feb 17 18:29:27 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:29:27 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s ocf-mysql-wss: 2016/02/17_18:29:28 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:29:28 INFO: MySQL is not running Feb 17 18:29:31 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:29:31 [7299] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s Feb 17 18:29:36 [7299] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:29:36 [7299] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(7585) = 0: Connection refused (111) Feb 17 18:29:36 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipc_us_ready: poll(fd 11) got POLLHUP Feb 17 18:29:36 [7299] node-3.test.domain.local pacemakerd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:29:36 [7299] node-3.test.domain.local pacemakerd: info: mcp_quorum_destroy: connection closed Feb 17 18:29:36 [7299] node-3.test.domain.local pacemakerd: debug: pcmk_shutdown_worker: crmd confirmed stopped Feb 17 18:29:36 [7299] node-3.test.domain.local pacemakerd: notice: stop_child: Stopping pengine: Sent -15 to process 7585 Feb 17 18:29:36 [7299] node-3.test.domain.local pacemakerd: debug: qb_ipc_us_ready: poll(fd 9) got POLLHUP Feb 17 18:29:36 [7299] node-3.test.domain.local pacemakerd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:29:36 [7299] node-3.test.domain.local pacemakerd: error: pcmk_cpg_dispatch: Connection to the CPG API failed: Library error (2) Feb 17 18:29:36 [7585] node-3.test.domain.local pengine: info: crm_signal_dispatch: Invoking handler for signal 15: Terminated Feb 17 18:29:36 [7299] node-3.test.domain.local pacemakerd: error: mcp_cpg_destroy: Connection destroyed Feb 17 18:29:36 [7585] node-3.test.domain.local pengine: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:29:36 [7299] node-3.test.domain.local pacemakerd: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:29:36 [7585] node-3.test.domain.local pengine: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:29:36 [7585] node-3.test.domain.local pengine: info: crm_xml_cleanup: Cleaning up memory from libxml2 ocf-mysql-wss: 2016/02/17_18:29:36 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:29:36 INFO: MySQL is not running Feb 17 18:29:38 [7495] node-3.test.domain.local pacemaker_remoted: info: crm_signal_dispatch: Invoking handler for signal 15: Terminated Feb 17 18:29:38 [7495] node-3.test.domain.local pacemaker_remoted: info: lrmd_shutdown: Terminating with 0 clients Feb 17 18:29:38 [7495] node-3.test.domain.local pacemaker_remoted: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:29:38 [7495] node-3.test.domain.local pacemaker_remoted: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:29:38 [7495] node-3.test.domain.local pacemaker_remoted: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:29:59 [4675] node-3.test.domain.local corosync notice [MAIN ] main Corosync Cluster Engine ('2.3.4'): started and ready to provide service. Feb 17 18:29:59 [4675] node-3.test.domain.local corosync info [MAIN ] main Corosync built-in features: dbus testagents rdma watchdog augeas pie relro bindnow Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize Token Timeout (3650 ms) retransmit timeout (357 ms) Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize token hold (275 ms) retransmits before loss (10 retrans) Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize join (60 ms) send_join (0 ms) consensus (3600 ms) merge (200 ms) Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize downcheck (1000 ms) fail to recv const (2500 msgs) Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize seqno unchanged const (30 rotations) Maximum network MTU 1401 Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize window size per rotation (50 messages) maximum messages per rotation (20 messages) Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize missed count const (5 messages) Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize send threads (2 threads) Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token expired timeout (357 ms) Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token problem counter (2000 ms) Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP threshold (10 problem count) Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP multicast threshold (100 problem count) Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP automatic recovery check timeout (1000 ms) Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP mode set to none. Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize heartbeat_failures_allowed (0) Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize max_network_delay (50 ms) Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0 Feb 17 18:29:59 [4675] node-3.test.domain.local corosync notice [TOTEM ] totemnet_instance_initialize Initializing transport (UDP/IP Unicast). Feb 17 18:29:59 [4675] node-3.test.domain.local corosync notice [TOTEM ] init_nss Initializing transmit/receive security (NSS) crypto: none hash: none Feb 17 18:29:59 [4675] node-3.test.domain.local corosync notice [TOTEM ] timer_function_netif_check_timeout The network interface is down. Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [TOTEM ] main_iface_change_fn Created or loaded sequence id 0.127.0.0.1 for this ring. Feb 17 18:29:59 [4675] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration map access [0] Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cmap [0] Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:29:59 [4675] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cmap Feb 17 18:29:59 [4675] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration service [1] Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cfg [1] Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:29:59 [4675] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cfg Feb 17 18:29:59 [4675] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync cluster closed process group service v1.01 [2] Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cpg [2] Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:29:59 [4675] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cpg Feb 17 18:29:59 [4675] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync profile loading service [4] Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on pload [4] Feb 17 18:29:59 [4675] node-3.test.domain.local corosync warning [WD ] setup_watchdog No Watchdog, try modprobe Feb 17 18:29:59 [4675] node-3.test.domain.local corosync info [WD ] wd_scan_resources no resources configured. Feb 17 18:29:59 [4675] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync watchdog service [7] Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on wd [7] Feb 17 18:29:59 [4675] node-3.test.domain.local corosync notice [QUORUM] quorum_exec_init_fn Using quorum provider corosync_votequorum Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_readconfig Reading configuration (runtime: 0) Feb 17 18:29:59 [4675] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_read_nodelist_configuration No nodelist defined or our node is not in the nodelist Feb 17 18:29:59 [4675] node-3.test.domain.local corosync crit [QUORUM] quorum_exec_init_fn Quorum provider: corosync_votequorum failed to initialize. Feb 17 18:29:59 [4675] node-3.test.domain.local corosync error [SERV ] corosync_service_defaults_link_and_init Service engine 'corosync_quorum' failed to load for reason 'configuration error: nodelist or quorum.expected_votes must be configured!' Feb 17 18:29:59 [4675] node-3.test.domain.local corosync error [MAIN ] _corosync_exit_error Corosync Cluster Engine exiting with status 20 at service.c:356. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [MAIN ] main Corosync Cluster Engine ('2.3.4'): started and ready to provide service. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync info [MAIN ] main Corosync built-in features: dbus testagents rdma watchdog augeas pie relro bindnow Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize Token Timeout (3650 ms) retransmit timeout (357 ms) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize token hold (275 ms) retransmits before loss (10 retrans) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize join (60 ms) send_join (0 ms) consensus (3600 ms) merge (200 ms) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize downcheck (1000 ms) fail to recv const (2500 msgs) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize seqno unchanged const (30 rotations) Maximum network MTU 1401 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize window size per rotation (50 messages) maximum messages per rotation (20 messages) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize missed count const (5 messages) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize send threads (2 threads) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token expired timeout (357 ms) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token problem counter (2000 ms) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP threshold (10 problem count) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP multicast threshold (100 problem count) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP automatic recovery check timeout (1000 ms) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP mode set to none. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize heartbeat_failures_allowed (0) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize max_network_delay (50 ms) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [TOTEM ] totemnet_instance_initialize Initializing transport (UDP/IP Unicast). Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [TOTEM ] init_nss Initializing transmit/receive security (NSS) crypto: none hash: none Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [TOTEM ] timer_function_netif_check_timeout The network interface [192.168.0.4] is now up. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] main_iface_change_fn Created or loaded sequence id 20.192.168.0.4 for this ring. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration map access [0] Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cmap [0] Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:34:21 [6267] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cmap Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration service [1] Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cfg [1] Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:34:21 [6267] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cfg Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync cluster closed process group service v1.01 [2] Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cpg [2] Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:34:21 [6267] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cpg Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync profile loading service [4] Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on pload [4] Feb 17 18:34:21 [6267] node-3.test.domain.local corosync warning [WD ] setup_watchdog No Watchdog, try modprobe Feb 17 18:34:21 [6267] node-3.test.domain.local corosync info [WD ] wd_scan_resources no resources configured. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync watchdog service [7] Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on wd [7] Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [QUORUM] quorum_exec_init_fn Using quorum provider corosync_votequorum Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_readconfig Reading configuration (runtime: 0) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_readconfig ev_tracking=0, ev_tracking_barrier = 0: expected_votes = 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=1, expected_votes=3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: Yes Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync vote quorum service v1.0 [5] Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on votequorum [5] Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:34:21 [6267] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: votequorum Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync cluster quorum service v0.1 [3] Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on quorum [3] Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:34:21 [6267] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: quorum Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [TOTEM ] totemudpu_member_add adding new UDPU member {192.168.0.6} Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [TOTEM ] totemudpu_member_add adding new UDPU member {192.168.0.4} Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [TOTEM ] totemudpu_member_add adding new UDPU member {192.168.0.5} Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_gather_enter entering GATHER state from 15(interface change). Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_commit_token_create Creating commit token because I am the rep. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] old_ring_state_save Saving state aru 0 high seq received 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] corosync_ring_id_store Storing new sequence id for ring 24 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_commit_enter entering COMMIT state. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token got commit token Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter entering RECOVERY state. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter position [0] member 192.168.0.4: Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter previous ring seq 20 rep 192.168.0.4 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter aru 0 high delivered 0 received flag 1 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter Did not need to originate any messages in recovery. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token got commit token Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token Sending initial ORF token Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token retrans flag count 4 token aru 0 install seq 0 aru 0 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] old_ring_state_reset Resetting old ring state Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] deliver_messages_from_recovery_to_regular recovery to regular 1-0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemudpu_member_set_active Marking UDPU member 192.168.0.4 active Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] member_object_joined Member joined: r(0) ip(192.168.0.4) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_operational_enter entering OPERATIONAL state. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [TOTEM ] memb_state_operational_enter A new membership (192.168.0.4:36) was formed. Members joined: 3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_gather_enter entering GATHER state from 11(merge during join). Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_commit_token_create Creating commit token because I am the rep. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] old_ring_state_save Saving state aru 0 high seq received 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] corosync_ring_id_store Storing new sequence id for ring 28 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_commit_enter entering COMMIT state. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token got commit token Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter entering RECOVERY state. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter TRANS [0] member 192.168.0.4: Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter position [0] member 192.168.0.4: Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter previous ring seq 24 rep 192.168.0.4 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter aru 0 high delivered 0 received flag 1 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter position [1] member 192.168.0.5: Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter previous ring seq 24 rep 192.168.0.5 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter aru 48 high delivered 48 received flag 1 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter Did not need to originate any messages in recovery. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token got commit token Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token Sending initial ORF token Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token retrans flag count 4 token aru 0 install seq 0 aru 0 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] old_ring_state_reset Resetting old ring state Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] deliver_messages_from_recovery_to_regular recovery to regular 1-0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totemudpu_member_set_active Marking UDPU member 192.168.0.5 active Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [MAIN ] member_object_joined Member joined: r(0) ip(192.168.0.5) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_state_operational_enter entering OPERATIONAL state. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [TOTEM ] memb_state_operational_enter A new membership (192.168.0.4:40) was formed. Members joined: 2 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[3]: votes: 1, expected: 3 flags: 8 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: Yes Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=1, expected_votes=3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [SYNC ] sync_service_build_handler service build for old ring - discarding Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [SYNC ] sync_barrier_handler Committing synchronization for corosync configuration map access Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_sync_activate My config version is 0 -> no action Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [CPG ] downlist_log comparing: sender r(0) ip(192.168.0.5) ; members(old:1 left:0) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [CPG ] downlist_log comparing: sender r(0) ip(192.168.0.4) ; members(old:0 left:0) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [CPG ] downlist_log chosen downlist: sender r(0) ip(192.168.0.5) ; members(old:1 left:0) Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_joinlist got joinlist message from node 0x2 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [SYNC ] sync_barrier_handler Committing synchronization for corosync cluster closed process group service v1.01 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [CPG ] joinlist_inform_clients joinlist_messages[0] group:crmd\x00, ip:r(0) ip(192.168.0.5) , pid:7403 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [CPG ] joinlist_inform_clients joinlist_messages[1] group:attrd\x00, ip:r(0) ip(192.168.0.5) , pid:7369 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [CPG ] joinlist_inform_clients joinlist_messages[2] group:stonithd\x00, ip:r(0) ip(192.168.0.5) , pid:7346 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [CPG ] joinlist_inform_clients joinlist_messages[3] group:cib\x00, ip:r(0) ip(192.168.0.5) , pid:7337 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [CPG ] joinlist_inform_clients joinlist_messages[4] group:pacemakerd\x00, ip:r(0) ip(192.168.0.5) , pid:7234 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 2 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[2]: votes: 1, expected: 3 flags: 1 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: Yes Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=2, expected_votes=3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 2 state=1, votes=1, expected=3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] get_lowest_node_id lowest node id: 2 us: 3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] get_highest_node_id highest node id: 3 us: 3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] are_we_quorate quorum regained, resuming activity Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 2 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[0]: votes: 0, expected: 0 flags: 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[3]: votes: 1, expected: 3 flags: 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=2, expected_votes=3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 2 state=1, votes=1, expected=3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] get_lowest_node_id lowest node id: 2 us: 3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] get_highest_node_id highest node id: 3 us: 3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[0]: votes: 0, expected: 0 flags: 0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [SYNC ] sync_barrier_handler Committing synchronization for corosync vote quorum service v1.0 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=2, expected_votes=3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 2 state=1, votes=1, expected=3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] get_lowest_node_id lowest node id: 2 us: 3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [VOTEQ ] get_highest_node_id highest node id: 3 us: 3 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [QUORUM] quorum_api_set_quorum This node is within the primary component and will provide service. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [QUORUM] log_view_list Members[2]: 3 2 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [QUORUM] send_library_notification sending quorum notification to (nil), length = 56 Feb 17 18:34:21 [6267] node-3.test.domain.local corosync notice [MAIN ] corosync_sync_completed Completed service synchronization, ready to provide service. Feb 17 18:34:21 [6267] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-18) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761319b690 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-19) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76131957f0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-19) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-19) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76131957f0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-19-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-19-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-19-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: notice: mcp_read_config: Configured corosync to accept connections from group 115: OK (1) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-18-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-18-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-18-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: notice: main: Starting Pacemaker 1.1.12 (Build: 561c4cf): generated-manpages agent-manpages ncurses libqb-logging libqb-ipc lha-fencing nagios heartbeat corosync-native snmp libesmtp acls Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: main: Maximum core file size is: 18446744073709551615 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-18) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-18) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: qb_ipcs_us_publish: server name: pacemakerd Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761319b690 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-18-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-18-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-18-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-18) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: cluster_connect_cfg: Our nodeid: 3 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-19) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761319bfa0, cpd=0x7f761319c534 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-20) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76131964a0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_procjoin got procjoin message from cluster node 0x3 (r(0) ip(192.168.0.4) ) for pid 6556 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-20-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-20-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-20-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: crm_get_peer: Created entry 46d20689-0030-4bca-8265-ad6d2a3eceea/0x1cce2b0 for node (null)/3 (1 total) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: crm_get_peer: Node 3 has uuid 3 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3] - corosync-cpg is now online Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: cluster_connect_quorum: Configuring Pacemaker to obtain quorum from Corosync Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-20) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-20) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76131964a0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-20-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-20-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-20-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-20) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7f761319aac0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7f761319aac0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7f761319aac0 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: notice: cluster_connect_quorum: Quorum acquired Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_trackstart got trackstart request on 0x7f761319aac0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_trackstart sending initial status to 0x7f761319aac0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QUORUM] send_library_notification sending quorum notification to 0x7f761319aac0, length = 56 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-22) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f7613190bc0 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-22-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-22-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-22-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 1 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: crm_get_peer: Created entry 23e1cc8e-b6cc-4b57-bae3-a79f9e2ba3d5/0x1ccf890 for node (null)/1 (2 total) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: crm_get_peer: Node 1 has uuid 1 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-22) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-22) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f7613190bc0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-22-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-22-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-22-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-22) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f7613190bc0 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-22-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-22-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-22-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-22) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-22) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f7613190bc0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-22-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-22-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-22-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-22) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f7613190bc0 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-22-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-22-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-22-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: crm_get_peer: Created entry d7929997-01b5-4bcc-b880-c25904a13c98/0x1ccfa80 for node (null)/2 (3 total) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: crm_get_peer: Node 2 has uuid 2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-22) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-22) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f7613190bc0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-22-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-22-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-22-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-21) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f7613190bc0 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: crm_get_peer: Node 3 is now known as node-3.test.domain.local Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-21) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f7613190bc0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761359d960 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: start_child: Using uid=108 and group=115 for process cib Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: start_child: Forked child 6655 for process cib Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-21) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761359d960 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.xml (digest: /var/lib/pacemaker/cib/cib.xml.sig) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761359d960 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000000100 (was 00000000000000000000000004000000) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: start_child: Forked child 6656 for process stonith-ng Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-21) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761359d960 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: validate_with_relaxng: Creating RNG parser context Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/root Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761359d960 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6656-22) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6656] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761359e170, cpd=0x7f761359e8c4 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000100100 (was 00000000000000000000000000000100) Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: start_child: Forked child 6668 for process lrmd Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-21) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761359d960 Feb 17 18:34:22 [6668] node-3.test.domain.local pacemaker_remoted: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/root Feb 17 18:34:22 [6668] node-3.test.domain.local pacemaker_remoted: info: qb_ipcs_us_publish: server name: lrmd Feb 17 18:34:22 [6668] node-3.test.domain.local pacemaker_remoted: info: main: Starting Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761359d960 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_procjoin got procjoin message from cluster node 0x3 (r(0) ip(192.168.0.4) ) for pid 6656 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000100110 (was 00000000000000000000000000100100) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: start_child: Using uid=108 and group=115 for process attrd Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: start_child: Forked child 6678 for process attrd Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6656-23) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6656] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f7613190d00 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-21) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761359d960 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: main: Starting up Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761359d960 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-24) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f7613198910, cpd=0x7f7613199014 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000101110 (was 00000000000000000000000000100110) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: start_child: Using uid=108 and group=115 for process pengine Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: start_child: Forked child 6692 for process pengine Feb 17 18:34:22 [6692] node-3.test.domain.local pengine: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster Feb 17 18:34:22 [6692] node-3.test.domain.local pengine: debug: main: Init server comms Feb 17 18:34:22 [6692] node-3.test.domain.local pengine: info: qb_ipcs_us_publish: server name: pengine Feb 17 18:34:22 [6692] node-3.test.domain.local pengine: info: main: Starting pengine Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-21) Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-21) state:2 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761359d960 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-23-header Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: info: crm_get_peer: Created entry f6c454c2-ec91-4a19-8418-e78ea5039529/0x1b3d300 for node (null)/3 (1 total) Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: info: crm_get_peer: Node 3 has uuid 3 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3] - corosync-cpg is now online Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: info: init_cs_connection_once: Connection to 'corosync': established Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: activateCibXml: Triggering CIB write for start op Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: startCib: CIB Initialization completed successfully Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761319a400 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6656-23) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6656-23) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f7613190d00 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6656-23) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6656] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f7613190d00 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-25) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761349f630, cpd=0x7f761349fbc4 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000111110 (was 00000000000000000000000000101110) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: start_child: Using uid=108 and group=115 for process crmd Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: start_child: Forked child 6707 for process crmd Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:34:22 [6707] node-3.test.domain.local crmd: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster Feb 17 18:34:22 [6707] node-3.test.domain.local crmd: notice: main: CRM Git Version: 561c4cf Feb 17 18:34:22 [6707] node-3.test.domain.local crmd: debug: crmd_init: Starting crmd Feb 17 18:34:22 [6707] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_STARTUP: [ state=S_STARTING cause=C_STARTUP origin=crmd_init ] Feb 17 18:34:22 [6707] node-3.test.domain.local crmd: info: do_log: FSA: Input I_STARTUP from crmd_init() received in state S_STARTING Feb 17 18:34:22 [6707] node-3.test.domain.local crmd: debug: do_startup: Registering Signal Handlers Feb 17 18:34:22 [6707] node-3.test.domain.local crmd: debug: do_startup: Creating CIB and LRM objects Feb 17 18:34:22 [6707] node-3.test.domain.local crmd: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:34:22 [6707] node-3.test.domain.local crmd: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:34:22 [6707] node-3.test.domain.local crmd: info: crm_ipc_connect: Could not establish cib_shm connection: Connection refused (111) Feb 17 18:34:22 [6707] node-3.test.domain.local crmd: debug: cib_native_signon_raw: Connection unsuccessful (0 (nil)) Feb 17 18:34:22 [6707] node-3.test.domain.local crmd: debug: cib_native_signon_raw: Connection to CIB failed: Transport endpoint is not connected Feb 17 18:34:22 [6707] node-3.test.domain.local crmd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a0f60 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_procjoin got procjoin message from cluster node 0x3 (r(0) ip(192.168.0.4) ) for pid 6678 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-21) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761319a400 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761319a400 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a56a0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_procjoin got procjoin message from cluster node 0x3 (r(0) ip(192.168.0.4) ) for pid 6655 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-23-header Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-23-header Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-23-header Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: info: crm_get_peer: Node 3 is now known as node-3.test.domain.local Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6656-23) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6656-23) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f7613190d00 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-23-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000111310 (was 00000000000000000000000000111110) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: main: Starting mainloop Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: pcmk_quorum_notification: Membership 40: quorum retained (2) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: pcmk_quorum_notification: Member[0] 3 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6656-23) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6656] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f7613190d00 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-21) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761319a400 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761319a400 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: crm_get_peer: Created entry bb16ff54-6074-418d-987f-b7497ccd8cd3/0x16a0ea0 for node (null)/3 (1 total) Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: crm_get_peer: Node 3 has uuid 3 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3] - corosync-cpg is now online Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: init_cs_connection_once: Connection to 'corosync': established Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: crm_get_peer: Created entry b3b37de9-2148-4854-8ae7-ab6f7ba5162d/0x1f718d0 for node (null)/3 (1 total) Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: crm_get_peer: Node 3 has uuid 3 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3] - corosync-cpg is now online Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: init_cs_connection_once: Connection to 'corosync': established Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a56a0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: notice: crm_update_peer_state: pcmk_quorum_notification: Node node-3.test.domain.local[3] - state is now member (was (null)) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: pcmk_quorum_notification: Member[1] 2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a0f60 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a56a0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-21) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761319a400 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-23-header Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-23-header Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: info: crm_ipc_connect: Could not establish cib_rw connection: Connection refused (111) Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: cib_native_signon_raw: Connection unsuccessful (0 (nil)) Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: cib_native_signon_raw: Connection to CIB failed: Transport endpoint is not connected Feb 17 18:34:22 [6656] node-3.test.domain.local stonithd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761319a400 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6656-23) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6656-23) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f7613190d00 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-23-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-23) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f7613190d00 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: crm_get_peer: Node 3 is now known as node-3.test.domain.local Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a56a0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761379f080 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-23-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-23-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-23-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: pcmk_quorum_notification: Obtaining name for new node 2 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: qb_ipcs_us_publish: server name: cib_ro Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: qb_ipcs_us_publish: server name: cib_rw Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: qb_ipcs_us_publish: server name: cib_shm Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: cib_init: Starting cib mainloop Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-21-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-21-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: pcmk_cpg_membership: Joined[0.0] cib.3 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-21-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: crm_get_peer: Node 3 is now known as node-3.test.domain.local Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-23) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-23) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f7613190d00 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-21) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761319a400 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761379f080 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761319a400 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-23) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a54f0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76137a1570 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-21) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761319a400 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-23-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-23-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-23-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: crm_get_peer: Created entry 566d5c1b-b704-4362-9132-27d8041e86e8/0x16a4170 for node (null)/2 (2 total) Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: crm_get_peer: Node 2 has uuid 2 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: pcmk_cpg_membership: Member[0.0] cib.2 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[2] - corosync-cpg is now online Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761319a400 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-23) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-23) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a54f0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-23-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: main: Cluster connection active Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: qb_ipcs_us_publish: server name: attrd Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: main: Accepting attribute updates Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: notice: main: Starting mainloop... Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: pcmk_cpg_membership: Joined[0.0] attrd.3 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76137a1570 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-23) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76137a1570 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-23-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-23-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-23-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: pcmk_cpg_membership: Member[0.1] cib.3 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: child_waitpid: wait(6782) = 0: Resource temporarily unavailable (11) Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: notice: crm_update_peer_state: pcmk_quorum_notification: Node (null)[2] - state is now member (was (null)) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: pcmk_quorum_notification: State of node (null)[1] is still unknown Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: info: crm_get_peer: Node 2 is now known as node-2.test.domain.local Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: crm_get_peer: Created entry f7e80a4b-9907-4621-bd38-44634c1a5bda/0x1f73170 for node (null)/2 (2 total) Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: crm_get_peer: Node 2 has uuid 2 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: pcmk_cpg_membership: Member[0.0] attrd.2 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[2] - corosync-cpg is now online Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761379f3a0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-23) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-23) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76137a1570 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-23-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-89.raw Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: write_cib_contents: Writing CIB to disk Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: write_cib_contents: Wrote version 0.155.0 of the CIB to disk (digest: ae87576247a47ebf73c1f1fc0288d287) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-21) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761319a400 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: write_cib_contents: Wrote digest ae87576247a47ebf73c1f1fc0288d287 to disk Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6655] node-3.test.domain.local cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.Thzt1A (digest: /var/lib/pacemaker/cib/cib.QNjK4r) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6655] node-3.test.domain.local cib: debug: write_cib_contents: Activating /var/lib/pacemaker/cib/cib.Thzt1A Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761379f3a0 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6556] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-2.test.domain.local now has process list: 00000000000000000000000000111310 (was 00000000000000000000000000000000) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6655) = 0: Resource temporarily unavailable (11) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6656) = 0: Resource temporarily unavailable (11) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6668) = 0: Resource temporarily unavailable (11) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6678) = 0: Resource temporarily unavailable (11) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6692) = 0: Resource temporarily unavailable (11) Feb 17 18:34:22 [6556] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6707) = 0: Resource temporarily unavailable (11) Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-23-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-23-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-23-header Feb 17 18:34:22 [6678] node-3.test.domain.local attrd: info: pcmk_cpg_membership: Member[0.1] attrd.3 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761319a400 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-23) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a5580 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6556-21) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-21) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761319a400 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6556-21-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-23) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-23) state:2 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a5580 Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-23-header Feb 17 18:34:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-23-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x16a4ec0 for uid=108 gid=115 pid=6707 id=1dde8be4-9eb4-48d4-b149-4b27f4a94be6 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-6707-11) Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [6707] Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: cib_common_callback_worker: Setting cib_refresh_notify callbacks for crmd (1dde8be4-9eb4-48d4-b149-4b27f4a94be6): on Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: cib_common_callback_worker: Setting cib_diff_notify callbacks for crmd (1dde8be4-9eb4-48d4-b149-4b27f4a94be6): on Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: do_cib_control: CIB connection established Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-21) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f76137a1570, cpd=0x7f761379efd4 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-23) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761379fd10 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_procjoin got procjoin message from cluster node 0x3 (r(0) ip(192.168.0.4) ) for pid 6707 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-23-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-23-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-23-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: crm_get_peer: Created entry 642b72a1-da0f-4807-b584-e019e2a018f7/0xf0b610 for node (null)/3 (1 total) Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: crm_get_peer: Node 3 has uuid 3 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3] - corosync-cpg is now online Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: init_cs_connection_once: Connection to 'corosync': established Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-23) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-23) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761379fd10 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-23-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-23-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-23-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-23) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76137a0a00 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-23-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-23-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-23-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: crm_get_peer: Node 3 is now known as node-3.test.domain.local Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: peer_update_callback: node-3.test.domain.local is now (null) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-23) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-23) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76137a0a00 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-23-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-23-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-23-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-23) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a4d40 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-23-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-23-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-23-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: cluster_connect_quorum: Configuring Pacemaker to obtain quorum from Corosync Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-23) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-23) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a4d40 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-23-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-23-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-23-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-23) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7f76134a4d40 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7f76134a4d40 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7f76134a4d40 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: notice: cluster_connect_quorum: Quorum acquired Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_trackstart got trackstart request on 0x7f76134a4d40 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_trackstart sending initial status to 0x7f76134a4d40 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QUORUM] send_library_notification sending quorum notification to 0x7f76134a4d40, length = 56 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a6660 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b10 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-28) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a3390 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-28-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-28-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-28-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 1 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: crm_get_peer: Created entry 3d28f174-5cb5-48c7-aff0-0e0b0cb2b9cb/0x110d580 for node (null)/1 (2 total) Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: crm_get_peer: Node 1 has uuid 1 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-28) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-28) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a3390 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-28-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-28-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-28-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b10 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6655] node-3.test.domain.local cib: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: info: crm_get_peer: Node 2 is now known as node-2.test.domain.local Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-28) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e740 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-27) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-28-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-28-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-28-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-28) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-28) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e740 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-28-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-28-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.0 2 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.1 (null) Feb 17 18:34:23 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=1 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status: Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-28-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/310, version=0.155.1) Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x172c560 for uid=0 gid=0 pid=6656 id=494ec8bb-1a9e-4390-8b8c-7dcdb560ac28 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-6656-12) Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [6656] Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: crm_get_peer: Created entry afa85642-cfaf-47ee-a21d-3deb179614b4/0x110d820 for node (null)/2 (3 total) Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: crm_get_peer: Node 2 has uuid 2 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-27) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: cib_common_callback_worker: Setting cib_diff_notify callbacks for crmd (494ec8bb-1a9e-4390-8b8c-7dcdb560ac28): on Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a6660 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: notice: setup_cib: Watching for stonith topology changes Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: qb_ipcs_us_publish: server name: stonith-ng Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: main: Starting stonithd mainloop Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: pcmk_cpg_membership: Joined[0.0] stonithd.3 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6656-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6656] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a6fd0 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-28) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e240 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-27-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-27-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-27-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: crm_get_peer: Created entry b0cbad84-b05f-44b9-9825-69dff0db554b/0x1b3f3f0 for node (null)/2 (2 total) Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: crm_get_peer: Node 2 has uuid 2 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: pcmk_cpg_membership: Member[0.0] stonithd.2 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[2] - corosync-cpg is now online Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: st_peer_update_callback: Broadcasting our uname because of node 2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6656-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6656-27) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a6fd0 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6656-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6656] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a6fd0 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-28-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-28-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-28-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 1 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-28) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-28) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e240 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-28-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-28-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-28-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-27-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-27-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-27-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6656-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6656-27) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a6fd0 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6656-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6656] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e240 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-27-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-27-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-27-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: pcmk_cpg_membership: Member[0.1] stonithd.3 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: init_cib_cache_cb: Updating device list from the cib: init Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: cib_devices_update: Updating devices to version 0.155.1 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: unpack_config: Stop all active resources: false Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: unpack_config: Default stickiness: 0 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: apply_system_health: Applying automated node health strategy: migrate-on-red Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: apply_system_health: Node node-1.test.domain.local has an combined system health of 0 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: apply_system_health: Node node-2.test.domain.local has an combined system health of 0 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: apply_system_health: Node node-3.test.domain.local has an combined system health of 0 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: info: crm_get_peer: Node 2 is now known as node-2.test.domain.local Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: st_peer_update_callback: Broadcasting our uname because of node 2 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-28) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a7410 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6656-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6656-27) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e240 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6656-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6656] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-28-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-28-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-28-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a2eb0 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-28) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-28) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a7410 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-28-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-28-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-27-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-27-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-28-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6656-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6656-27) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a2eb0 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a6fd0 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6656-28) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6656] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e170 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-27) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a6fd0 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: do_ha_control: Connected to the cluster Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: do_lrm_control: Connecting to the LRM Feb 17 18:34:23 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section nodes to master (origin=local/crmd/3) Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: lrmd_ipc_connect: Connecting to lrmd Feb 17 18:34:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: crm_client_new: Connecting 0xcf2b10 for uid=108 gid=115 pid=6707 id=33b25540-7772-40d4-b8de-ed3f21e9e5ee Feb 17 18:34:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: handle_new_connection: IPC credentials authenticated (6668-6707-7) Feb 17 18:34:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: qb_ipcs_shm_connect: connecting to client [6707] Feb 17 18:34:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed register operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: do_lrm_control: LRM connection established Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: do_started: Delaying start, no membership data (0000000000100000) Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: register_fsa_input_adv: Stalling the FSA pending further input: source=do_started cause=C_FSA_INTERNAL data=(nil) queue=0 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: s_crmd_fsa: Exiting the FSA: queue=0, fsa_actions=0x2, stalled=true Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: pcmk_quorum_notification: Membership 40: quorum retained (2) Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: pcmk_quorum_notification: Member[0] 3 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-28-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-28-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-28-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a6ea0 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6656-28) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6656-28) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e170 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-28-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-28-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-28-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:23 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-3.test.domain.local/crmd/3, version=0.155.1) Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6656-28) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6656] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e140 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: notice: crm_update_peer_state: pcmk_quorum_notification: Node node-3.test.domain.local[3] - state is now member (was (null)) Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: peer_update_callback: node-3.test.domain.local is now member (was (null)) Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: pcmk_quorum_notification: Member[1] 2 Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-28-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-28-header Feb 17 18:34:23 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-28-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-27) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-27) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a6ea0 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6656-28) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6656-28) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e140 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6656-28-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6656-28-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6656-28-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: pcmk_quorum_notification: Obtaining name for new node 2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: notice: crm_update_peer_state: pcmk_quorum_notification: Node (null)[2] - state is now member (was (null)) Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: pcmk_quorum_notification: State of node (null)[1] is still unknown Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: post_cache_update: Updated cache after membership event 40. Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: post_cache_update: post_cache_update added action A_ELECTION_CHECK to the FSA Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: do_started: Delaying start, Config not read (0000000000000040) Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: register_fsa_input_adv: Stalling the FSA pending further input: source=do_started cause=C_FSA_INTERNAL data=(nil) queue=0 Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: s_crmd_fsa: Exiting the FSA: queue=0, fsa_actions=0x200000002, stalled=true Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: config_query_callback: Call 4 : Parsing CIB options Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: config_query_callback: Shutdown escalation occurs after: 1200000ms Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: config_query_callback: Checking for expired actions every 190000ms Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: do_started: Init server comms Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: qb_ipcs_us_publish: server name: crmd Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: notice: do_started: The local CRM is operational Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: do_election_check: Ignore election check: we not in an election Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_STARTING cause=C_FSA_INTERNAL origin=do_started ] Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: info: do_log: FSA: Input I_PENDING from do_started() received in state S_STARTING Feb 17 18:34:23 [6707] node-3.test.domain.local crmd: notice: do_state_transition: State transition S_STARTING -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_started ] Feb 17 18:34:23 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: do_cl_join_query: Querying for a DC Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=19 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: info: pcmk_cpg_membership: Joined[0.0] crmd.3 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: info: pcmk_cpg_membership: Member[0.0] crmd.2 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[2] - corosync-cpg is now online Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: info: pcmk_cpg_membership: Member[0.1] crmd.3 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: info: crm_get_peer: Node 2 is now known as node-2.test.domain.local Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: info: peer_update_callback: node-2.test.domain.local is now member Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: handle_request: Raising I_JOIN_OFFER: join-2 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ] Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: info: update_dc: Set DC to node-2.test.domain.local (3.0.9) Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: join_query_callback: Respond to join offer join-2 from node-2.test.domain.local Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: handle_request: Raising I_JOIN_OFFER: join-2 Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ] Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:24 [6707] node-3.test.domain.local crmd: debug: te_connect_stonith: Attempting connection to fencing daemon... Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_process_replace: Digest matched on replace from node-2.test.domain.local: 8c4abd030194beba0d5b27ae07087b8a Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_process_replace: Replaced 0.155.1 with 0.155.436 from node-2.test.domain.local Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: __xml_diff_object: Moved node_state@uname (1 -> 4) Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.1 2 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.436 (null) Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=436, @cib-last-written=Wed Feb 17 14:21:54 2016, @dc-uuid=2 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status: Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']: @in_ccm=true, @uname=node-3.test.domain.local, @join=down, @expected=member Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']: Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']: Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: activateCibXml: Triggering CIB write for cib_replace op Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_replace_notify: Replaced: 0.155.1 -> 0.155.436 from node-2.test.domain.local Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_replace operation for section 'all': OK (rc=0, origin=node-2.test.domain.local/crmd/312, version=0.155.436) Feb 17 18:34:24 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.436 to 0.155.1 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-2.test.domain.local/crmd/313, version=0.155.436) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-2.test.domain.local/crmd/314, version=0.155.436) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-90.raw Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: write_cib_contents: Writing CIB to disk Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: write_cib_contents: Wrote version 0.155.0 of the CIB to disk (digest: 41e83ddd2a00fe6d9db6a5db9f5e57a4) Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: write_cib_contents: Wrote digest 41e83ddd2a00fe6d9db6a5db9f5e57a4 to disk Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.nJfC7s (digest: /var/lib/pacemaker/cib/cib.zvBYcs) Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: write_cib_contents: Activating /var/lib/pacemaker/cib/cib.nJfC7s Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_delete op for //node_state[@uname='node-2.test.domain.local']/lrm (/cib/status/node_state[1]/lrm) Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.436 2 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.437 (null) Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: -- /cib/status/node_state[@id='2']/lrm[@id='2'] Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=437 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:24 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.437 to 0.155.436 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='node-2.test.domain.local']/lrm: OK (rc=0, origin=node-2.test.domain.local/crmd/315, version=0.155.437) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.437 2 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.438 (null) Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=438 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_lrm_query_internal Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']: Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:24 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.438 to 0.155.437 Feb 17 18:34:24 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/316, version=0.155.438) Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: crm_client_new: Connecting 0x1ba4110 for uid=108 gid=115 pid=6707 id=1f2093e1-de07-4405-ac50-29847d3e7d31 Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: handle_new_connection: IPC credentials authenticated (6656-6707-10) Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: qb_ipcs_shm_connect: connecting to client [6707] Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: stonith_command: Processing register 11 from crmd.6707 ( 0) Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: stonith_command: Processed register from crmd.6707: OK (0) Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: stonith_api_signon: Connection to STONITH successful Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: stonith_command: Processing st_notify 12 from crmd.6707 ( 0) Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: handle_request: Setting st_notify_disconnect callbacks for crmd.6707 (1f2093e1-de07-4405-ac50-29847d3e7d31): ON Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: stonith_command: Processed st_notify from crmd.6707: OK (0) Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: stonith_command: Processing st_notify 13 from crmd.6707 ( 0) Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: handle_request: Setting st_notify_fence callbacks for crmd.6707 (1f2093e1-de07-4405-ac50-29847d3e7d31): ON Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: stonith_command: Processed st_notify from crmd.6707: OK (0) Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: join_query_callback: Respond to join offer join-2 from node-2.test.domain.local Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: do_cib_replaced: Updating the CIB after a replace: DC=false Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: handle_request: Raising I_JOIN_RESULT: join-2 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ] Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: do_cl_join_finalize_respond: Confirming join join-2: join_ack_nack Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: do_cl_join_finalize_respond: join-2: Join complete. Sending local LRM status to node-2.test.domain.local Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: info: erase_status_tag: Deleting xpath: //node_state[@uname='node-3.test.domain.local']/transient_attributes Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: info: update_attrd_helper: Connecting to attrd... 5 retries remaining Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x1f73d70 for uid=108 gid=115 pid=6707 id=a71e2e69-7cb4-4cef-a032-e2e68316c84b Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-6707-9) Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [6707] Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_delete operation for section //node_state[@uname='node-3.test.domain.local']/transient_attributes to master (origin=local/crmd/8) Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: attrd_update_delegate: Sent update: terminate=(null) for node-3.test.domain.local Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: attrd_update_delegate: Sent update: shutdown=0 for node-3.test.domain.local Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crmd: terminate= Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for terminate Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null) Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crmd: shutdown=0 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for shutdown Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: (null), Stored: (null) Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of shutdown is 0 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: shutdown (0) Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation shutdown=0: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349da50 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-28) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1f40 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-28-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-28-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-28-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ] Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: info: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ] Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_delete op for //node_state[@uname='node-3.test.domain.local']/transient_attributes (/cib/status/node_state[2]/transient_attributes) Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.438 2 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.439 (null) Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: -- /cib/status/node_state[@id='3']/transient_attributes[@id='3'] Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=439 Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.439 to 0.155.438 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='node-3.test.domain.local']/transient_attributes: OK (rc=0, origin=node-3.test.domain.local/crmd/8, version=0.155.439) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-28) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-28) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1f40 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-28-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-28-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-28-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: notice: attrd_local_callback: Sending full refresh (origin=crmd) Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: shutdown (0) Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation shutdown=0: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349da50 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349da50 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349da50 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349da50 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349da50 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: handle_request: Raising I_JOIN_OFFER: join-2 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_NOT_DC cause=C_HA_MESSAGE origin=route_message ] Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: notice: do_state_transition: State transition S_NOT_DC -> S_PENDING [ input=I_JOIN_OFFER cause=C_HA_MESSAGE origin=route_message ] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: info: update_dc: Unset DC. Was node-2.test.domain.local Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: info: update_dc: Set DC to node-2.test.domain.local (3.0.9) Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: erase_xpath_callback: Deletion of "//node_state[@uname='node-3.test.domain.local']/transient_attributes": OK (rc=0) Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: join_query_callback: Respond to join offer join-2 from node-2.test.domain.local Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/671, version=0.155.439) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/673, version=0.155.439) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_replace: Digest matched on replace from node-2.test.domain.local: af8684bc937d8623ec1b85b61020d795 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_replace: Replaced 0.155.439 with 0.155.439 from node-2.test.domain.local Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: activateCibXml: Triggering CIB write for cib_replace op Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_replace operation for section 'all': OK (rc=0, origin=node-2.test.domain.local/crmd/318, version=0.155.439) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d9e0 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-2.test.domain.local/crmd/319, version=0.155.439) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d9e0 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d9e0 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-2.test.domain.local/crmd/320, version=0.155.439) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: handle_request: Raising I_JOIN_RESULT: join-2 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ] Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: do_cl_join_finalize_respond: Confirming join join-2: join_ack_nack Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-91.raw Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: write_cib_contents: Writing CIB to disk Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d9e0 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: write_cib_contents: Wrote version 0.155.0 of the CIB to disk (digest: bf51af5ae7aa887394743a3711c94ad3) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: write_cib_contents: Wrote digest bf51af5ae7aa887394743a3711c94ad3 to disk Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.jUWmNG (digest: /var/lib/pacemaker/cib/cib.pI0M8J) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: write_cib_contents: Activating /var/lib/pacemaker/cib/cib.jUWmNG Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: do_cl_join_finalize_respond: join-2: Join complete. Sending local LRM status to node-2.test.domain.local Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ] Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: info: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING Feb 17 18:34:25 [6707] node-3.test.domain.local crmd: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ] Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: notice: attrd_local_callback: Sending full refresh (origin=crmd) Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: shutdown (0) Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation shutdown=0: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_delete op for //node_state[@uname='node-2.test.domain.local']/lrm (/cib/status/node_state[1]/lrm) Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.439 2 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.440 (null) Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: -- /cib/status/node_state[@id='2']/lrm[@id='2'] Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=440 Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.440 to 0.155.439 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='node-2.test.domain.local']/lrm: OK (rc=0, origin=node-2.test.domain.local/crmd/321, version=0.155.440) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.440 2 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.441 (null) Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=441 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']: Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.441 to 0.155.440 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/322, version=0.155.441) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_delete op for //node_state[@uname='node-3.test.domain.local']/lrm (/cib/status/node_state[2]/lrm) Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.441 2 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.442 (null) Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: -- /cib/status/node_state[@id='3']/lrm[@id='3'] Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=442 Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.442 to 0.155.441 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='node-3.test.domain.local']/lrm: OK (rc=0, origin=node-2.test.domain.local/crmd/323, version=0.155.442) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.442 2 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.443 (null) Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=443 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']: @crm-debug-origin=do_lrm_query_internal, @join=member Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']: Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/324, version=0.155.443) Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.443 to 0.155.442 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/675, version=0.155.443) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-2.test.domain.local/crmd/327, version=0.155.443) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.443 2 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.444 (null) Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=444 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']: @crm-debug-origin=do_state_transition Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/328, version=0.155.444) Feb 17 18:34:25 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.444 to 0.155.443 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: crm_get_peer: Node 2 is now known as node-2.test.domain.local Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section cib: OK (rc=0, origin=node-2.test.domain.local/crmd/329, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dad0 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for rabbit_get_alarms_timeouts Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation rabbit_get_alarms_timeouts=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dad0 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e5c0 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/677, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for fail-count-p_mysql Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation fail-count-p_mysql=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e5c0 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/679, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349daf0 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/681, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for free_swap Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation free_swap=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349daf0 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/683, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349daf0 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for last-failure-p_neutron-l3-agent Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation last-failure-p_neutron-l3-agent=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349daf0 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e900 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/685, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for gtid Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation gtid=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e900 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e900 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/687, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for rabbit_list_channels_timeouts Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation rabbit_list_channels_timeouts=: cib not connected Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e900 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e8d0 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/689, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for fail-count-p_neutron-l3-agent Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation fail-count-p_neutron-l3-agent=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e8d0 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e720 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/691, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for cpu_cores Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation cpu_cores=: cib not connected Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e720 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dc70 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/693, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for arch Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation arch=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dc70 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b10 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/695, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for ram_total Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation ram_total=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/697, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for ram_free Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation ram_free=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/699, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for rabbit_list_queues_timeouts Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation rabbit_list_queues_timeouts=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/701, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for cpu_load Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation cpu_load=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/703, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for root_free Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation root_free=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d940 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/705, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for var_lib_mysql_free Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation var_lib_mysql_free=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d940 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/707, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for last-failure-p_mysql Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation last-failure-p_mysql=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d940 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/709, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for cpu_info Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation cpu_info=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d940 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/711, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for os Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation os=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/713, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation shutdown=0: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d940 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/715, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for master-p_rabbitmq-server Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation master-p_rabbitmq-server=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d940 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e2e0 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/717, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for master-p_conntrackd Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation master-p_conntrackd=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e2e0 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b10 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/719, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for cpu_speed Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation cpu_speed=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a67a0 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/721, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for probe_complete Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation probe_complete=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a67a0 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a67a0 Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/723, version=0.155.444) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for var_log_free Feb 17 18:34:25 [6678] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation var_log_free=: cib not connected Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a67a0 Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'sysinfo_node-1.test.domain.local' not found (0 active resources) Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'sysinfo_node-1.test.domain.local' to the rsc list (1 active resources) Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=5:76:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=sysinfo_node-1.test.domain.local_monitor_0 Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=5, reply=1, notify=0, exit=4202464 Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:sysinfo_node-1.test.domain.local action:monitor call_id:5 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7844) = 0: Resource temporarily unavailable (11) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-1.test.domain.local_monitor_0:7844 - exited with rc=7 Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: sysinfo_node-1.test.domain.local_monitor_0:7844:stderr [ rm: cannot remove '/var/run//SysInfo-sysinfo_node-1.test.domain.local': No such file or directory ] Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-1.test.domain.local_monitor_0:7844:stdout [ -- empty -- ] Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-1.test.domain.local action:monitor call_id:5 pid:7844 exit-code:7 exec-time:6ms queue-time:0ms Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_vrouter' not found (1 active resources) Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_vrouter:1' not found (1 active resources) Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_vrouter' to the rsc list (2 active resources) Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=6:76:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_vrouter_monitor_0 Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=10, reply=1, notify=0, exit=4202464 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource sysinfo_node-1.test.domain.local after monitor op complete (interval=0) Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_vrouter action:monitor call_id:10 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed SysInfo_meta-data_0 process 7851 exited with rc=0 Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/11) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation sysinfo_node-1.test.domain.local_monitor_0: not running (node=node-3.test.domain.local, call=5, rc=7, cib-update=11, confirmed=true) Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: node-3.test.domain.local-sysinfo_node-1.test.domain.local_monitor_0:5 [ rm: cannot remove '/var/run//SysInfo-sysinfo_node-1.test.domain.local': No such file or directory\n ] Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'sysinfo_node-1.test.domain.local' with monitor op Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.444 2 Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.445 (null) Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=445 Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']: @crm-debug-origin=do_update_resource Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/11, version=0.155.445) Feb 17 18:34:26 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.445 to 0.155.444 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 ocf-ns_vrouter: 2016/02/17_18:34:26 DEBUG: p_vrouter: check_ns(): recieved netns list: Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'vip__management' not found (2 active resources) Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'vip__management' to the rsc list (3 active resources) Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=7:76:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=vip__management_monitor_0 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=14, reply=1, notify=0, exit=4202464 Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:vip__management action:monitor call_id:14 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7853) = 0: Resource temporarily unavailable (11) Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__management_monitor_0:7869 - exited with rc=7 Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__management_monitor_0:7869:stderr [ -- empty -- ] Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__management_monitor_0:7869:stdout [ -- empty -- ] Feb 17 18:34:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__management action:monitor call_id:14 pid:7869 exit-code:7 exec-time:13ms queue-time:0ms Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource vip__management after monitor op complete (interval=0) Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_IPaddr2_meta-data_0 process 7876 exited with rc=0 Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation vip__management_monitor_0: not running (node=node-3.test.domain.local, call=14, rc=7, cib-update=12, confirmed=true) Feb 17 18:34:26 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'vip__management' with monitor op Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/12) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.445 2 Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.446 (null) Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=446 Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.449 2 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.450 (null) Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=450 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:27 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.451 to 0.155.450 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/21, version=0.155.451) Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_speed'] does not exist Feb 17 18:34:27 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for cpu_speed=(null) passed Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_conntrackd'] does not exist Feb 17 18:34:27 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for master-p_conntrackd=(null) passed Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='probe_complete'] does not exist Feb 17 18:34:27 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for probe_complete=(null) passed Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] does not exist Feb 17 18:34:27 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for var_log_free=(null) passed Feb 17 18:34:27 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 21 for shutdown=0 passed ocf-ns_haproxy: 2016/02/17_18:34:27 DEBUG: p_haproxy: check_ns(): recieved netns list: ocf-ns_vrouter: 2016/02/17_18:34:27 DEBUG: Adding route on the host system to 192.168.2.0/24: 240.0.0.6 ocf-ns_vrouter: 2016/02/17_18:34:27 DEBUG: Adding route on the host system to 192.168.1.0/24: 240.0.0.6 ocf-ns_haproxy: 2016/02/17_18:34:27 ERROR: sysctl: cannot stat /proc/sys/net/ipv4/ip_nonlocal_bind: No such file or directory ocf-ns_vrouter: 2016/02/17_18:34:27 DEBUG: Adding route on the host system to 192.168.0.0/24: 240.0.0.6 ocf-ns_haproxy: 2016/02/17_18:34:27 DEBUG: p_haproxy: get_ns(): added netns haproxy and set up lo ocf-ns_haproxy: 2016/02/17_18:34:27 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:34:27 INFO: haproxy daemon is not running Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7853) = 0: Resource temporarily unavailable (11) Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_0:8989 - exited with rc=7 Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_0:8989:stderr [ -- empty -- ] Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_0:8989:stdout [ -- empty -- ] Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:36 pid:8989 exit-code:7 exec-time:191ms queue-time:0ms Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_haproxy after monitor op complete (interval=0) Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_haproxy_meta-data_0 process 9180 exited with rc=0 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: get_rsc_restart_list: Attr dummy is not reloadable Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/17) Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_haproxy_monitor_0: not running (node=node-3.test.domain.local, call=36, rc=7, cib-update=17, confirmed=true) Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_haproxy' with monitor op Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.451 2 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.452 (null) Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=452 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/17, version=0.155.452) Feb 17 18:34:27 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.452 to 0.155.451 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_vrouter: 2016/02/17_18:34:27 DEBUG: Adding route on the host system to 10.109.1.0/24: 240.0.0.6 Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_0:7853 - exited with rc=0 Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_0:7853:stderr [ -- empty -- ] Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_0:7853:stdout [ -- empty -- ] Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:10 pid:7853 exit-code:0 exec-time:886ms queue-time:1ms Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_vrouter after monitor op complete (interval=0) Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_vrouter_meta-data_0 process 9251 exited with rc=0 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_vrouter_monitor_0: ok (node=node-3.test.domain.local, call=10, rc=0, cib-update=18, confirmed=true) Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_vrouter' with monitor op Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/18) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.452 2 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.453 (null) Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=453 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/18, version=0.155.453) Feb 17 18:34:27 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.453 to 0.155.452 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=19:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_vrouter_monitor_30000 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=37, reply=1, notify=0, exit=4202464 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_vrouter action:monitor call_id:37 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for vip__vrouter_pub_start_0 Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=28:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=vip__vrouter_pub_start_0 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=38, reply=1, notify=0, exit=4202464 Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:vip__vrouter_pub action:start call_id:38 ocf-ns_vrouter: 2016/02/17_18:34:27 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.453 2 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.454 (null) Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=454 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_vrouter']/lrm_rsc_op[@id='p_vrouter_last_0']: @operation_key=p_vrouter_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=20:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;20:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=242, @last-run=1455734067, @last-rc-change=1455734067, @exec-time= Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/334, version=0.155.454) Feb 17 18:34:27 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.454 to 0.155.453 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_vrouter: 2016/02/17_18:34:27 INFO: net.ipv4.ip_forward = 1 Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:9315 - exited with rc=0 Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:9315:stderr [ -- empty -- ] Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:9315:stdout [ -- empty -- ] Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:9315 exit-code:0 exec-time:118ms queue-time:0ms Feb 17 18:34:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9326) = 0: Resource temporarily unavailable (11) Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_vrouter after monitor op complete (interval=30000) Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_vrouter_monitor_30000: ok (node=node-3.test.domain.local, call=37, rc=0, cib-update=19, confirmed=false) Feb 17 18:34:27 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_vrouter' with monitor op Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/19) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:27 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:27 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.454 2 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.455 (null) Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=455 Feb 17 18:34:27 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_vrouter']: word_size:263168 Feb 17 18:34:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:29 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.458 2 Feb 17 18:34:29 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.459 (null) Feb 17 18:34:29 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=459 Feb 17 18:34:29 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='vip__vrouter']/lrm_rsc_op[@id='vip__vrouter_last_0']: @operation_key=vip__vrouter_start_0, @operation=start, @transition-key=30:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;30:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=39, @rc-code=0, @last-run=1455734067, @last-rc-change=1455734067, @exec-time=1348 Feb 17 18:34:29 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/20, version=0.155.459) Feb 17 18:34:29 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.459 to 0.155.458 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=31:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=vip__vrouter_monitor_5000 Feb 17 18:34:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=40, reply=1, notify=0, exit=4202464 Feb 17 18:34:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:vip__vrouter action:monitor call_id:40 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_start_0:9326 - exited with rc=0 Feb 17 18:34:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_start_0:9326:stderr [ -- empty -- ] Feb 17 18:34:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_start_0:9326:stdout [ Device "v_vrouter_pub" does not exist. ] Feb 17 18:34:29 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:vip__vrouter_pub action:start call_id:38 pid:9326 exit-code:0 exec-time:1522ms queue-time:0ms Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource vip__vrouter_pub after start op complete (interval=0) Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation vip__vrouter_pub_start_0: ok (node=node-3.test.domain.local, call=38, rc=0, cib-update=21, confirmed=true) Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-vip__vrouter_pub_start_0:38 [ Device "v_vrouter_pub" does not exist.\n ] Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'vip__vrouter_pub' with start op Feb 17 18:34:29 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/21) Feb 17 18:34:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10431) = 0: Resource temporarily unavailable (11) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:29 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:29 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.459 2 Feb 17 18:34:29 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.460 (null) Feb 17 18:34:29 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=460 Feb 17 18:34:29 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='vip__vrouter_pub']/lrm_rsc_op[@id='vip__vrouter_pub_last_0']: @operation_key=vip__vrouter_pub_start_0, @operation=start, @transition-key=28:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;28:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=38, @rc-code=0, @last-run=1455734067, @last-rc-change=1455734067, @exec-time=1522 Feb 17 18:34:29 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/21, version=0.155.460) Feb 17 18:34:29 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.460 to 0.155.459 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:29 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=29:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=vip__vrouter_pub_monitor_5000 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=41, reply=1, notify=0, exit=4202464 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:vip__vrouter_pub action:monitor call_id:41 Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:31 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:31 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:31 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:31 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:31 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:31 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:31 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:31 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:31 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:31 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:31 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:31 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.460 2 Feb 17 18:34:31 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.461 (null) Feb 17 18:34:31 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=461 Feb 17 18:34:31 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='vip__management']: word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.461 2 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.462 (null) Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=462 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='vip__vrouter']: word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_conntrackd_start_0 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=34:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_start_0 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=42, reply=1, notify=0, exit=4202464 Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_conntrackd action:start call_id:42 ocf-ns_IPaddr2: 2016/02/17_18:34:32 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.814ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.537ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.537ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:10439 - exited with rc=0 Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:10439:stderr [ -- empty -- ] Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:10439:stdout [ -- empty -- ] Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:10439 exit-code:0 exec-time:3142ms queue-time:0ms Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource vip__vrouter_pub after monitor op complete (interval=5000) Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation vip__vrouter_pub_monitor_5000: ok (node=node-3.test.domain.local, call=41, rc=0, cib-update=23, confirmed=false) Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'vip__vrouter_pub' with monitor op Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/23) Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10665) = 0: Resource temporarily unavailable (11) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.462 2 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.463 (null) Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=463 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='vip__vrouter_pub']: word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a6660 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a6660 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a6660 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_haproxy_start_0 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=63:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_haproxy_start_0 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a6660 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=43, reply=1, notify=0, exit=4202464 Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_haproxy action:start call_id:43 ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_conntrackd: 2016/02/17_18:34:32 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff ocf-ns_haproxy: 2016/02/17_18:34:32 INFO: haproxy daemon is not running ocf-ns_conntrackd: 2016/02/17_18:34:32 INFO: Starting conntrackd ocf-ns_haproxy: 2016/02/17_18:34:32 INFO: stdin: is not a tty ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: Creating host interface: hapr-host and namespace interface: hapr-ns ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: Moving interface: hapr-ns to namespace: haproxy ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: Bringing up host interface: hapr-host ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: Setting host interface: hapr-host IP to: 240.0.0.1/30 ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: Bringing up the namespace interface: hapr-ns ocf-ns_conntrackd: 2016/02/17_18:34:32 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-11189-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [11189] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f76134a6660, cpd=0x7f76134a4134 Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f76134a6660 Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-6268-11189-26-header Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-6268-11189-26-header Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-6268-11189-26-header Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-11189-26-header Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-11189-26-header Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-11189-26-header Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:34:32 [11189] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-11189-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-11189-26) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f76134a6660 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-11189-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-11189-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-11189-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-11189-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [11189] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-11189-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-11189-26) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-11189-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-11189-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-11189-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x172ea50 for uid=0 gid=0 pid=11200 id=cee1c37c-3547-43f5-b110-900847ac6b00 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-11200-14) Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [11200] Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201d930 for uid=0 gid=0 pid=11200 id=d81d9272-ac29-4b0b-9922-0981b3e326bb Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11200-11) Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11200] Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: master-p_conntrackd=100 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 100, Current: (null), Stored: (null) Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of master-p_conntrackd is 100 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_conntrackd (100) Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_conntrackd'] does not exist Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 29: master-p_conntrackd=100 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/29) Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: master-p_conntrackd=100 for node-3.test.domain.local Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: info: main: Update master-p_conntrackd=100 sent via attrd Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-11200-14-header Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-11200-14-header Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-11200-14-header Feb 17 18:34:32 [11200] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11200-11) Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11200-11) state:2 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11200-11-header Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11200-11-header Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11200-11-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-11200-14) Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-11200-14) state:2 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-11200-14-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-11200-14-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-11200-14-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dbc0 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.463 2 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.464 (null) Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=464 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/29, version=0.155.464) Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:32 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.464 to 0.155.463 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 29 for master-p_conntrackd=100 passed Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dbc0 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-11208-27) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [11208] ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: Setting namespace interface: hapr-ns IP to: 240.0.0.2/30 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761349d940, cpd=0x7f761349ded4 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f761349d940 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/725, version=0.155.464) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-6268-11208-27-header Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-6268-11208-27-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-6268-11208-27-header Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-11208-27) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-11208-27) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761349d940 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-11208-27-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-11208-27-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-11208-27-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-11208-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [11208] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-11208-26-header Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-11208-26-header Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-11208-26-header Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:34:32 [11208] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-11208-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-11208-26) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-11208-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-11208-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-11208-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x172ea50 for uid=0 gid=0 pid=11218 id=ef5e4387-0aa0-462b-8b6b-daf0773e43ae Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-11218-14) Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [11218] Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201d930 for uid=0 gid=0 pid=11218 id=f26f6fa0-ada9-4288-a908-546cac556bbc Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11218-11) Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11218] Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: master-p_conntrackd=100 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100 Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: master-p_conntrackd=100 for node-3.test.domain.local Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: info: main: Update master-p_conntrackd=100 sent via attrd Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-11218-14-header Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-11218-14-header Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-11218-14-header Feb 17 18:34:32 [11218] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11218-11) Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11218-11) state:2 Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11218-11-header Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11218-11-header Feb 17 18:34:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11218-11-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-11218-14) Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-11218-14) state:2 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-11218-14-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-11218-14-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-11218-14-header Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:10665 - exited with rc=0 Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:10665:stderr [ -- empty -- ] Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:10665:stdout [ vrouter ] Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:10665:stdout [ mgmt-conntrd ] Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:10665:stdout [ vrouter ] Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:10665:stdout [ mgmt-conntrd ] Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_conntrackd action:start call_id:42 pid:10665 exit-code:0 exec-time:386ms queue-time:0ms Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_conntrackd after start op complete (interval=0) Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_start_0: ok (node=node-3.test.domain.local, call=42, rc=0, cib-update=24, confirmed=true) Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_conntrackd_start_0:42 [ vrouter\n mgmt-conntrd\nvrouter\n mgmt-conntrd\n ] Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_conntrackd' with start op Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/24) Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10684) = 0: Resource temporarily unavailable (11) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.464 2 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.465 (null) Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=465 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_conntrackd']/lrm_rsc_op[@id='p_conntrackd_last_0']: @operation_key=p_conntrackd_start_0, @operation=start, @transition-key=34:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;34:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=42, @rc-code=0, @last-run=1455734072, @last-rc-change=1455734072, @exec-time=386 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/24, version=0.155.465) Feb 17 18:34:32 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.465 to 0.155.464 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: Creating default route inside the namespace to 240.0.0.1 with metric 10000 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=200:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_notify_0 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=44, reply=1, notify=0, exit=4202464 Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_conntrackd action:notify call_id:44 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10684) = 0: Resource temporarily unavailable (11) Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:11230 - exited with rc=0 Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:11230:stderr [ -- empty -- ] Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:11230:stdout [ -- empty -- ] Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_conntrackd action:notify call_id:44 pid:11230 exit-code:0 exec-time:15ms queue-time:0ms Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_conntrackd after notify op complete (interval=0) Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_conntrackd_notify_0 from 200:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455734072-7 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_notify_0: ok (node=node-3.test.domain.local, call=44, rc=0, cib-update=0, confirmed=true) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header ocf-ns_haproxy: 2016/02/17_18:34:32 INFO: net.ipv4.conf.hapr-host.rp_filter = 2 ocf-ns_haproxy: 2016/02/17_18:34:32 INFO: net.ipv4.conf.all.rp_filter = 2 ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: Adding route on the host system to 10.109.0.0/24: 240.0.0.2 ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: Adding route on the host system to 192.168.2.0/24: 240.0.0.2 ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: Adding route on the host system to 192.168.1.0/24: 240.0.0.2 ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: Adding route on the host system to 192.168.0.0/24: 240.0.0.2 ocf-ns_haproxy: 2016/02/17_18:34:32 DEBUG: Adding route on the host system to 10.109.1.0/24: 240.0.0.2 ocf-ns_haproxy: 2016/02/17_18:34:32 INFO: Started haproxy daemon. Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_start_0:10684 - exited with rc=0 Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_start_0:10684:stderr [ -- empty -- ] Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_start_0:10684:stdout [ -- empty -- ] Feb 17 18:34:32 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_haproxy action:start call_id:43 pid:10684 exit-code:0 exec-time:728ms queue-time:0ms Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_haproxy after start op complete (interval=0) Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_haproxy_start_0: ok (node=node-3.test.domain.local, call=43, rc=0, cib-update=25, confirmed=true) Feb 17 18:34:32 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_haproxy' with start op Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/25) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:32 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.465 2 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.466 (null) Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=466 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_haproxy']/lrm_rsc_op[@id='p_haproxy_last_0']: @operation_key=p_haproxy_start_0, @operation=start, @transition-key=63:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;63:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=43, @rc-code=0, @last-run=1455734072, @last-rc-change=1455734072, @exec-time=728 Feb 17 18:34:32 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/25, version=0.155.466) Feb 17 18:34:32 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.466 to 0.155.465 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:32 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.466 2 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.467 (null) Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=467 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_haproxy']/lrm_rsc_op[@id='p_haproxy_last_0']: @operation_key=p_haproxy_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=61:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;61:77:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=243, @last-run=1455734067, @last-rc-change=1455734067, @exec-time= Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/339, version=0.155.467) Feb 17 18:34:33 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.467 to 0.155.466 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=72:78:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_haproxy_monitor_30000 Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=45, reply=1, notify=0, exit=4202464 Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_haproxy action:monitor call_id:45 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_mysql' not found (8 active resources) Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_mysql:1' not found (8 active resources) Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_mysql' to the rsc list (9 active resources) Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=10:78:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_mysql_monitor_0 Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=50, reply=1, notify=0, exit=4202464 Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_mysql action:monitor call_id:50 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header ocf-ns_haproxy: 2016/02/17_18:34:33 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:34:33 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.467 2 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.468 (null) Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=468 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_haproxy']: Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201d930 for uid=0 gid=0 pid=11562 id=3975c2be-5bb7-4abc-a110-79a4e17dd44f Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11562-11) Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11562] Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: master-p_rabbitmq-server=0 Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: (null), Stored: (null) Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of master-p_rabbitmq-server is 0 Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_rabbitmq-server (0) Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: master-p_rabbitmq-server=0 for node-3.test.domain.local Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: info: main: Update master-p_rabbitmq-server=0 sent via attrd Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-11562-14-header Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-11562-14-header Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-11562-14-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_rabbitmq-server'] does not exist Feb 17 18:34:33 [11562] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-11562-14) Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-11562-14) state:2 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-11562-14-header Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 31: master-p_rabbitmq-server=0 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-11562-14-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-11562-14-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/31) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dc50 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11562-11) Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11562-11) state:2 Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11562-11-header Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11562-11-header Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11562-11-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dc50 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() lrmd: 2016/02/17_18:34:33 DEBUG: p_rabbitmq-server: monitor: role: Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.469 2 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.470 (null) Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=470 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:34:33 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.470 to 0.155.469 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/31, version=0.155.470) Feb 17 18:34:33 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 31 for master-p_rabbitmq-server=0 passed Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 lrmd: 2016/02/17_18:34:33 DEBUG: p_rabbitmq-server: monitor: result: 7 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/727, version=0.155.470) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header lrmd: 2016/02/17_18:34:33 DEBUG: p_rabbitmq-server: monitor: action end. Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11306) = 0: Resource temporarily unavailable (11) Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_monitor_0:11356 - exited with rc=7 Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:11356:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:11356:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:11356:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:11356:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:11356:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:11356:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_monitor_0:11356:stdout [ -- empty -- ] Feb 17 18:34:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_rabbitmq-server action:monitor call_id:56 pid:11356 exit-code:7 exec-time:611ms queue-time:0ms Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_rabbitmq-server after monitor op complete (interval=0) Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-11592-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [11592] Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761349cf10, cpd=0x7f761349d974 Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f761349cf10 Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-6268-11592-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-11592-26) Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-6268-11592-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-11592-26) state:2 Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-6268-11592-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-11592-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-11592-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-11592-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-11592-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [11592] Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-11592-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-11592-26) Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-11592-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-11592-26) state:2 Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-11592-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:33 [11592] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-11592-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-11592-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-11592-26-header Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed rabbitmq-server_meta-data_0 process 11576 exited with rc=0 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/27) Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_rabbitmq-server_monitor_0: not running (node=node-3.test.domain.local, call=56, rc=7, cib-update=27, confirmed=true) Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: node-3.test.domain.local-p_rabbitmq-server_monitor_0:56 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n ] Feb 17 18:34:33 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_rabbitmq-server' with monitor op Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.470 2 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.471 (null) Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=471 Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/27, version=0.155.471) Feb 17 18:34:33 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.471 to 0.155.470 Feb 17 18:34:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:34:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:34:38 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 2 retries left Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:38 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:38 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:38 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 34567e1f9bdadc785b4926d78f3a98c0 for 0.155.471 (0x1733cf0 0) Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:38 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_IPaddr2: 2016/02/17_18:34:40 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.643ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.671ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.537ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.543ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:34:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11306) = 0: Interrupted system call (4) Feb 17 18:34:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11627 - exited with rc=0 Feb 17 18:34:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11627:stderr [ -- empty -- ] Feb 17 18:34:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11627:stdout [ -- empty -- ] Feb 17 18:34:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:11627 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:34:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11640) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:34:40 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.866ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.550ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.547ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:34:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11306) = 0: Resource temporarily unavailable (11) Feb 17 18:34:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:11640 - exited with rc=0 Feb 17 18:34:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:11640:stderr [ -- empty -- ] Feb 17 18:34:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:11640:stdout [ -- empty -- ] Feb 17 18:34:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:11640 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:34:43 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 1 retries left Feb 17 18:34:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:34:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:34:48 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:34:48 INFO: MySQL is not running Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_0:11306 - exited with rc=7 Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_0:11306:stderr [ -- empty -- ] Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_0:11306:stdout [ -- empty -- ] Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_mysql action:monitor call_id:50 pid:11306 exit-code:7 exec-time:15045ms queue-time:0ms Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_mysql after monitor op complete (interval=0) Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11816) = 0: Resource temporarily unavailable (11) Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11829) = 0: Resource temporarily unavailable (11) Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed mysql-wss_meta-data_0 process 11904 exited with rc=0 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/28) Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_mysql_monitor_0: not running (node=node-3.test.domain.local, call=50, rc=7, cib-update=28, confirmed=true) Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_mysql' with monitor op Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.471 2 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.472 (null) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=472 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/28, version=0.155.472) Feb 17 18:34:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.472 to 0.155.471 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-ns_IPaddr2: 2016/02/17_18:34:48 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.685ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.727ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.554ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.544ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11816 - exited with rc=0 Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11816:stderr [ -- empty -- ] Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11816:stdout [ -- empty -- ] Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:11816 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11829) = 0: Resource temporarily unavailable (11) Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_dns' not found (10 active resources) Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_dns:1' not found (10 active resources) Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_dns' to the rsc list (11 active resources) Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=12:79:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_dns_monitor_0 Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=61, reply=1, notify=0, exit=4202464 Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_dns action:monitor call_id:61 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_heat-engine' not found (11 active resources) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_heat-engine:1' not found (11 active resources) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_heat-engine' to the rsc list (12 active resources) Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=13:79:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_heat-engine_monitor_0 Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=66, reply=1, notify=0, exit=4202464 Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_heat-engine action:monitor call_id:66 ocf-ns_dns: 2016/02/17_18:34:48 INFO: dnsmasq daemon is not running Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11829) = 0: Resource temporarily unavailable (11) Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_0:11917 - exited with rc=7 Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_0:11917:stderr [ -- empty -- ] Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_0:11917:stdout [ -- empty -- ] Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:61 pid:11917 exit-code:7 exec-time:27ms queue-time:0ms Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_dns after monitor op complete (interval=0) Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11920) = 0: Resource temporarily unavailable (11) Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_dns_meta-data_0 process 11934 exited with rc=0 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/29) Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_dns_monitor_0: not running (node=node-3.test.domain.local, call=61, rc=7, cib-update=29, confirmed=true) Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_dns' with monitor op Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.472 2 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.473 (null) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=473 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/29, version=0.155.473) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.473 to 0.155.472 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-heat-engine: 2016/02/17_18:34:48 DEBUG: Create pid file dir: /var/run/resource-agents/heat-engine and chown to heat ocf-ns_IPaddr2: 2016/02/17_18:34:48 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.783ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.538ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.543ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:11829 - exited with rc=0 Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:11829:stderr [ -- empty -- ] Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:11829:stdout [ -- empty -- ] Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:11829 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11920) = 0: Resource temporarily unavailable (11) ocf-heat-engine: 2016/02/17_18:34:48 INFO: OpenStack Heat Engine is not running Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_0:11920 - exited with rc=7 Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_0:11920:stderr [ -- empty -- ] Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_0:11920:stdout [ -- empty -- ] Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:66 pid:11920 exit-code:7 exec-time:46ms queue-time:0ms Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=213:79:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_notify_0 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=67, reply=1, notify=0, exit=4202464 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_heat-engine after monitor op complete (interval=0) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:48 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_conntrackd action:notify call_id:67 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed heat-engine_meta-data_0 process 11966 exited with rc=0 Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_heat-engine_monitor_0: not running (node=node-3.test.domain.local, call=66, rc=7, cib-update=30, confirmed=true) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/30) Feb 17 18:34:48 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_heat-engine' with monitor op Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.473 2 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.474 (null) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=474 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201d930 for uid=0 gid=0 pid=12185 id=e8a91a12-17a1-48f0-a39f-8acbf89114fc Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12185-11) Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12185] Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_list_channels_timeouts=0 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: (null), Stored: (null) Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of rabbit_list_channels_timeouts is 0 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: rabbit_list_channels_timeouts (0) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit_list_channels_timeouts'] does not exist Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/33) Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 33: rabbit_list_channels_timeouts=0 Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_list_channels_timeouts=0 for node-3.test.domain.local Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: info: main: Update rabbit_list_channels_timeouts=0 sent via attrd Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-12185-14-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-12185-14) Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-12185-14-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-12185-14) state:2 Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-12185-14-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-12185-14-header Feb 17 18:34:48 [12185] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-12185-14-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-12185-14-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12185-11) Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12185-11) state:2 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12185-11-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12185-11-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12185-11-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dbf0 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.474 2 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.475 (null) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=475 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/33, version=0.155.475) Feb 17 18:34:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.475 to 0.155.474 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1764c50 for uid=0 gid=0 pid=12188 id=b4608386-4e2f-4f32-a78c-8ff91e6aa109 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-12188-14) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12188] Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 33 for rabbit_list_channels_timeouts=0 passed Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201d930 for uid=0 gid=0 pid=12188 id=4697a33e-beda-4539-a6e0-7fbd8c71329c Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12188-11) Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12188] Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dbf0 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/729, version=0.155.475) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_get_alarms_timeouts=0 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: (null), Stored: (null) Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of rabbit_get_alarms_timeouts is 0 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: rabbit_get_alarms_timeouts (0) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit_get_alarms_timeouts'] does not exist Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 35: rabbit_get_alarms_timeouts=0 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/35) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_get_alarms_timeouts=0 for node-3.test.domain.local Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: info: main: Update rabbit_get_alarms_timeouts=0 sent via attrd Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-12188-14-header Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-12188-14-header Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-12188-14-header Feb 17 18:34:48 [12188] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dbe0 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12188-11) Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12188-11) state:2 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12188-11-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12188-11-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12188-11-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.475 2 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.476 (null) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=476 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/35, version=0.155.476) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-12188-14) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-12188-14) state:2 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-12188-14-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-12188-14-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-12188-14-header Feb 17 18:34:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.476 to 0.155.475 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 35 for rabbit_get_alarms_timeouts=0 passed Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dbe0 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/731, version=0.155.476) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1764c50 for uid=0 gid=0 pid=12192 id=0d069b3e-3663-4b9a-82e3-f798cb100dd1 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-12192-14) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12192] Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201d930 for uid=0 gid=0 pid=12192 id=eaf2bca8-70d8-4549-979c-3288496a536f Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12192-11) Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12192] Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_list_queues_timeouts=0 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: (null), Stored: (null) Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of rabbit_list_queues_timeouts is 0 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: rabbit_list_queues_timeouts (0) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit_list_queues_timeouts'] does not exist Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_list_queues_timeouts=0 for node-3.test.domain.local Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: info: main: Update rabbit_list_queues_timeouts=0 sent via attrd Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-12192-14-header Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-12192-14-header Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-12192-14-header Feb 17 18:34:48 [12192] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-12192-14) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-12192-14) state:2 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-12192-14-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-12192-14-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-12192-14-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 37: rabbit_list_queues_timeouts=0 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12192-11) Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12192-11) state:2 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12192-11-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12192-11-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12192-11-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/37) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 lrmd: 2016/02/17_18:34:48 INFO: p_rabbitmq-server: start: Deleting start time attribute Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349ddd0 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.476 2 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.477 (null) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=477 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/37, version=0.155.477) Feb 17 18:34:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.477 to 0.155.476 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 37 for rabbit_list_queues_timeouts=0 passed Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349ddd0 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/733, version=0.155.477) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1764c50 for uid=0 gid=0 pid=12199 id=07331984-5c6e-418e-bb94-61114f2934a7 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-12199-14) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12199] Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201d930 for uid=0 gid=0 pid=12199 id=37b303b2-009e-4305-8486-94f423494695 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12199-11) Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12199] Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit-start-time= Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for rabbit-start-time Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null) Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit-start-time=(null) for node-3.test.domain.local Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: info: main: Update rabbit-start-time= sent via attrd Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-12199-14-header Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-12199-14-header Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-12199-14-header Feb 17 18:34:48 [12199] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-12199-14) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-12199-14) state:2 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-12199-14-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-12199-14-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-12199-14-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12199-11) Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12199-11) state:2 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12199-11-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12199-11-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12199-11-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header lrmd: 2016/02/17_18:34:48 INFO: p_rabbitmq-server: start: Deleting master attribute Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1764c50 for uid=0 gid=0 pid=12206 id=04e9b6bb-0b2a-4c42-aca6-1eb334d994dc Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-12206-14) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12206] Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201d930 for uid=0 gid=0 pid=12206 id=4d66df05-ec32-41c5-9b8b-51f1629112cc Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12206-11) Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12206] Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit-master= Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for rabbit-master Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null) Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit-master=(null) for node-3.test.domain.local Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: info: main: Update rabbit-master= sent via attrd Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-12206-14) Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-12206-14) state:2 Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-12206-14-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-12206-14-header Feb 17 18:34:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-12206-14-header Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-12206-14-header Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-12206-14-header Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-12206-14-header Feb 17 18:34:48 [12206] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12206-11) Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12206-11) state:2 Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12206-11-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12206-11-header Feb 17 18:34:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12206-11-header lrmd: 2016/02/17_18:34:48 INFO: p_rabbitmq-server: start: RMQ going to start. lrmd: 2016/02/17_18:34:48 INFO: p_rabbitmq-server: start_rmq_server_app(): begin. lrmd: 2016/02/17_18:34:49 INFO: p_rabbitmq-server: start_rmq_server_app(): blocked access to RMQ port lrmd: 2016/02/17_18:34:50 INFO: get_status() failed with code 2. Command output: Error: unable to connect to node 'rabbit@messaging-node-3': nodedown DIAGNOSTICS =========== attempted to contact: ['rabbit@messaging-node-3'] rabbit@messaging-node-3: * connected to epmd (port 4369) on messaging-node-3 * epmd reports: node 'rabbit' not running at all other nodes on messaging-node-3: ['rabbitmq-cli-12255'] * suggestion: start the node current node details: - node name: 'rabbitmq-cli-12255@node-3' - home dir: /var/lib/rabbitmq - cookie hash: soeIWU2jk2YNseTyDSlsEA== lrmd: 2016/02/17_18:34:50 INFO: p_rabbitmq-server: start_rmq_server_app(): RMQ-runtime (beam) not started, starting... lrmd: 2016/02/17_18:34:51 INFO: p_rabbitmq-server: start_rmq_server_app(): RMQ-server app not started, starting... lrmd: 2016/02/17_18:34:51 INFO: p_rabbitmq-server: try_to_start_rmq_app(): begin. lrmd: 2016/02/17_18:34:51 INFO: p_rabbitmq-server: try_to_start_rmq_app(): Execute start_app with timeout: 90 lrmd: 2016/02/17_18:34:51 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl start_app >>/var/log/rabbitmq/startup_log 2>&1 Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: Init 10 + 9 ticks at 1455734093 (100 tps) Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 2.130000 (full: 2.13 0.71 0.25 3/511 12661) Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: throttle_handle_load: Noticable CPU load detected: 2.130000 Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: throttle_timer_cb: New throttle mode: 0001 (was ffffffff) Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: info: throttle_send_command: Updated throttle state to 0001 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:53 [6707] node-3.test.domain.local crmd: debug: throttle_update: Host node-3.test.domain.local supports a maximum of 4 jobs and throttle mode 0001. New job limit is 2 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:34:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:34:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:34:53 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:34:53 INFO: MySQL is not running ocf-mysql-wss: 2016/02/17_18:34:53 INFO: Create socket dir: /var/run/mysqld and chown to mysql:mysql ocf-mysql-wss: 2016/02/17_18:34:53 INFO: Create PID dir: /var/run/resource-agents/mysql-wss and chown to mysql:mysql Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:53 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 0af5b5b18bd8f41425f6359cb0fb4973 for 0.155.477 (0x1733cf0 0) Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_IPaddr2: 2016/02/17_18:34:56 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.684ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.706ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.530ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.540ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:34:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:34:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11983) = 0: Interrupted system call (4) Feb 17 18:34:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:12662 - exited with rc=0 Feb 17 18:34:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:12662:stderr [ -- empty -- ] Feb 17 18:34:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:12662:stdout [ -- empty -- ] Feb 17 18:34:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:12662 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:34:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12681) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:34:56 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.874ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.538ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:34:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:34:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11983) = 0: Resource temporarily unavailable (11) Feb 17 18:34:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:12681 - exited with rc=0 Feb 17 18:34:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:12681:stderr [ -- empty -- ] Feb 17 18:34:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:12681:stdout [ -- empty -- ] Feb 17 18:34:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:12681 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:34:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:34:57 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:34:57 INFO: net.ipv4.ip_forward = 1 Feb 17 18:34:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:34:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11983) = 0: Resource temporarily unavailable (11) Feb 17 18:34:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:13069 - exited with rc=0 Feb 17 18:34:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:13069:stderr [ -- empty -- ] Feb 17 18:34:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:13069:stdout [ -- empty -- ] Feb 17 18:34:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:13069 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:34:58 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:34:58 ERROR: MySQL is not running ocf-mysql-wss: 2016/02/17_18:34:58 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1 ocf-mysql-wss: 2016/02/17_18:34:58 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1764c50 for uid=0 gid=0 pid=13116 id=8fba413c-815e-4042-bbaf-9917722d6b3a Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-13116-14) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13116] Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201d930 for uid=0 gid=0 pid=13116 id=b7595632-80f2-4304-9ec7-ad1531fd2387 Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-13116-11) Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13116] Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1, Current: (null), Stored: (null) Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of gtid is 71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: gtid (71dae69f-d579-11e5-a980-12eca8fb60f4:-1) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] does not exist Feb 17 18:34:58 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/39) Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 39: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:-1 for node-3.test.domain.local Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: info: main: Update gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:-1 sent via attrd Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-13116-14) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-13116-14) state:2 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13116-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13116-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13116-14-header Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13116-14-header Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13116-14-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13116-14-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [13116] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dbf0 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:58 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.477 2 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.478 (null) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=478 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:34:58 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/39, version=0.155.478) Feb 17 18:34:58 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.478 to 0.155.477 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dbf0 Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 39 for gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:-1 passed Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-13116-11) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-13116-11) state:2 Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-13116-11-header Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-13116-11-header Feb 17 18:34:58 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-13116-11-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [13118] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:34:58 [13118] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:34:58 [13118] node-3.test.domain.local crm_node: debug: main: Attempting to process -q command for cluster type: corosync Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-13118-27) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [13118] Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7f76134a1cc0 Feb 17 18:34:58 [13118] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [13118] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:58 [13118] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:58 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/735, version=0.155.478) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7f76134a1cc0 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7f76134a1cc0 Feb 17 18:34:58 [13118] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:58 [13118] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-request-6268-13118-27-header Feb 17 18:34:58 [13118] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-response-6268-13118-27-header Feb 17 18:34:58 [13118] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-event-6268-13118-27-header Feb 17 18:34:58 [13118] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-13118-27) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-13118-27) state:2 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7f76134a1cc0 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-6268-13118-27-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-6268-13118-27-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-6268-13118-27-header Feb 17 18:34:58 [13127] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:34:58 [13127] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:34:58 [13127] node-3.test.domain.local crm_node: debug: main: Attempting to process -l command for cluster type: corosync Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cde900 for uid=0 gid=0 pid=13127 id=d9a78040-b19e-4000-802f-337d271ff714 Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-13127-12) Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [13127] Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [13127] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [13127] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [13127] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [13127] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-13127-12) Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-13127-12) state:2 Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-13127-12-header Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-13127-12-header Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-13127-12-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x167efa0 for uid=0 gid=0 pid=13131 id=f24a4aba-cef4-47b9-9487-aa58a06a46a5 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-13131-14) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13131] Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: unpack_config: Stop all active resources: false Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: unpack_config: Default stickiness: 0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:0 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13131-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-13131-14) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-13131-14) state:2 Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13131-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13131-14-header Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13131-14-header Feb 17 18:34:58 [13131] node-3.test.domain.local crm_resource: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13131-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13131-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x167efa0 for uid=0 gid=0 pid=13136 id=3b67be26-a4dc-4dbf-b631-ed84bc20b415 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-13136-14) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13136] Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13136] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13136] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13136] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13136] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/configuration/crm_config//cluster_property_set//nvpair[@name='galera_master_timeout'] does not exist Feb 17 18:34:58 [13136] node-3.test.domain.local crm_attribute: info: main: Read galera_master_timeout=300 Feb 17 18:34:58 [13136] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:58 [13136] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-13136-14) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-13136-14) state:2 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13136-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13136-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13136-14-header Feb 17 18:34:58 [13136] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13136-14-header Feb 17 18:34:58 [13136] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13136-14-header Feb 17 18:34:58 [13136] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13136-14-header Feb 17 18:34:58 [13136] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 ocf-mysql-wss: 2016/02/17_18:34:58 INFO: Checking if Primary Component Feb 17 18:34:58 [13144] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:34:58 [13144] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:34:58 [13144] node-3.test.domain.local crm_node: debug: main: Attempting to process -p command for cluster type: corosync Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=13144 id=52000873-6ff3-4f1d-954a-2934b93e1ea1 Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-13144-12) Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [13144] Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [13144] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [13144] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [13144] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [13144] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-13144-12) Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-13144-12) state:2 Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-13144-12-header Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-13144-12-header Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-13144-12-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x167efa0 for uid=0 gid=0 pid=13149 id=adc552b3-0259-4c07-8137-684a9e47db80 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-13149-14) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13149] Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-2.test.domain.local to 2 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='2']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: info: main: Read gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:20770 Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13149-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-13149-14) Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13149-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-13149-14) state:2 Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13149-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13149-14-header Feb 17 18:34:58 [13149] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13149-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13149-14-header ocf-mysql-wss: 2016/02/17_18:34:58 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:20770 ocf-mysql-wss: 2016/02/17_18:34:58 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:20770 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x167efa0 for uid=0 gid=0 pid=13166 id=4c471f7c-7cbc-4253-827d-70b34c974f50 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-13166-14) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13166] Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[7]) Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: info: main: Read gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13166-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-13166-14) Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13166-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-13166-14) state:2 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13166-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13166-14-header Feb 17 18:34:58 [13166] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13166-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13166-14-header ocf-mysql-wss: 2016/02/17_18:34:58 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1 ocf-mysql-wss: 2016/02/17_18:34:58 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x167efa0 for uid=0 gid=0 pid=13186 id=262752f9-a35d-41b3-9cc9-c23791227462 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-13186-14) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13186] Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13186-14-header Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13186-14-header Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13186-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-13186-14) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-13186-14) state:2 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: validate_with_relaxng: Creating RNG parser context Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13186-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13186-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13186-14-header Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: cib_file_signon: crm_simulate: Opened connection to local file '/var/lib/pacemaker/cib/shadow.13186' Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: cib_file_perform_op_delegate: cib_query on (null) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: unpack_config: Stop all active resources: false Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: unpack_config: Default stickiness: 0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: apply_system_health: Applying automated node health strategy: migrate-on-red Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-1.test.domain.local has an combined system health of 0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-2.test.domain.local has an combined system health of 0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-3.test.domain.local has an combined system health of 0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-1.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_vrouter [p_vrouter] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_print: vip__management (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter_pub (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_print: vip__public (ocf::fuel:ns_IPaddr2): Stopped Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_conntrackd [p_conntrackd] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: short_print: Slaves: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_haproxy [p_haproxy] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_mysql [p_mysql] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_dns [p_dns] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_heat-engine [p_heat-engine] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-plugin-openvswitch-agent [p_neutron-plugin-openvswitch-agent] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-l3-agent [p_neutron-l3-agent] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-dhcp-agent [p_neutron-dhcp-agent] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-metadata-agent [p_neutron-metadata-agent] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-3.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-2.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_ping_vip__public [ping_vip__public] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_ntp [p_ntp] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__management: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 36 times on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999964 more times on node-2.test.domain.local before being forced off Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 36 times on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999964 more times on node-2.test.domain.local before being forced off Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 36 times on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999964 more times on node-2.test.domain.local before being forced off Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:0 has failed 4 times on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter_pub: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource sysinfo_node-1.test.domain.local are unavailable, unclean or shutting down (node-1.test.domain.local: 0, 1000000) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for sysinfo_node-1.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: Resource sysinfo_node-1.test.domain.local cannot run anywhere Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_vrouter:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_vrouter:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_vrouter:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_vrouter:2 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: Resource p_vrouter:2 cannot run anywhere Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_vrouter instances of a possible 3 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: clone_p_haproxy: Rolling back scores from vip__public Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_haproxy:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_haproxy:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_haproxy:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_haproxy:2 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: Resource p_haproxy:2 cannot run anywhere Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_haproxy instances of a possible 3 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__management Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter_pub Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource vip__public are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for vip__public Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: Resource vip__public cannot run anywhere Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: p_conntrackd:0: Rolling back scores from vip__vrouter Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_conntrackd:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_conntrackd:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: p_conntrackd:2: Rolling back scores from vip__vrouter_pub Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_conntrackd:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_conntrackd:2 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: Resource p_conntrackd:2 cannot run anywhere Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_conntrackd instances of a possible 3 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: master_p_conntrackd: Breaking dependency loop at master_p_conntrackd Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:1 master score: 1000000 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_conntrackd:1 (Slave node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:2 master score: 0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:0 master score: -1000000 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: master_color: master_p_conntrackd: Promoted 1 instances of a possible 1 to master Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_mysql:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_mysql:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_mysql:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_mysql:2 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: Resource p_mysql:2 cannot run anywhere Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_mysql instances of a possible 3 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_rabbitmq-server:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_rabbitmq-server:2 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: Resource p_rabbitmq-server:2 cannot run anywhere Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_rabbitmq-server instances of a possible 3 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:0 master score: 0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_rabbitmq-server:0 (Stopped node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:2 master score: 0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:1 master score: -1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: master_color: master_p_rabbitmq-server: Promoted 1 instances of a possible 1 to master Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:0 with p_vrouter:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_dns:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:1 with p_vrouter:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_dns:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:2 with p_vrouter:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_dns:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_dns:2 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: Resource p_dns:2 cannot run anywhere Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_dns instances of a possible 3 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_heat-engine:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_heat-engine:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_heat-engine:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_heat-engine:2 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: Resource p_heat-engine:2 cannot run anywhere Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_heat-engine instances of a possible 3 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-plugin-openvswitch-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-plugin-openvswitch-agent:2 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-plugin-openvswitch-agent:2 cannot run anywhere Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-plugin-openvswitch-agent instances of a possible 3 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-l3-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-l3-agent:2 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-l3-agent:2 cannot run anywhere Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-l3-agent instances of a possible 3 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-dhcp-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-dhcp-agent:2 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-dhcp-agent:2 cannot run anywhere Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-dhcp-agent instances of a possible 3 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-metadata-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-metadata-agent:2 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-metadata-agent:2 cannot run anywhere Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-metadata-agent instances of a possible 3 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to sysinfo_node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to sysinfo_node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to ping_vip__public:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to ping_vip__public:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource ping_vip__public:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for ping_vip__public:2 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: Resource ping_vip__public:2 cannot run anywhere Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_ping_vip__public instances of a possible 3 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_ntp:0 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_ntp:1 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_ntp:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_ntp:2 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: native_color: Resource p_ntp:2 cannot run anywhere Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_ntp instances of a possible 3 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-plugin-openvswitch-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-l3-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-dhcp-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-metadata-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing sysinfo_node-3.test.domain.local on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing sysinfo_node-2.test.domain.local on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing ping_vip__public:1 on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_ntp:1 on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_conntrackd Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:0 on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:1 on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_rabbitmq-server Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:0 on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:1 on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:0 on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:1 on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:0 on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:1 on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:0 on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:1 on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:0 on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:1 on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:0 on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:1 on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (15s) for sysinfo_node-3.test.domain.local on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (15s) for sysinfo_node-2.test.domain.local on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:0 on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:1 on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:0 on node-2.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:1 on node-3.test.domain.local Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_conntrackd:1 (Slave -> Master node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:0 (node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:1 (node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_rabbitmq-server:0 (node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_rabbitmq-server:0 (Stopped -> Master node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_rabbitmq-server:1 (node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_dns:0 (node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_dns:1 (node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_heat-engine:0 (node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_heat-engine:1 (node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:0 (node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:1 (node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:0 (node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:1 (node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:0 (node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:1 (node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:0 (node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:1 (node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start sysinfo_node-3.test.domain.local (node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start sysinfo_node-2.test.domain.local (node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:0 (node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:1 (node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:0 (node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:1 (node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:0 (aka. p_dns) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:0 (aka. p_dns) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:0 (aka. p_heat-engine) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:0 (aka. p_heat-engine) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: debug: cib_file_signoff: Signing out of the CIB Service Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: cib_file_signoff: Wrote CIB to /var/lib/pacemaker/cib/shadow.13186 Feb 17 18:34:58 [13186] node-3.test.domain.local crm_simulate: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x167efa0 for uid=0 gid=0 pid=13192 id=95764cd4-7efd-4881-a7ca-8903a35d7147 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-13192-14) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13192] Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-13192-14) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-13192-14) state:2 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13192-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13192-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13192-14-header Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13192-14-header Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13192-14-header Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13192-14-header Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: validate_with_relaxng: Creating RNG parser context Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: cib_file_signon: crm_simulate: Opened connection to local file '/var/lib/pacemaker/cib/shadow.13192' Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: cib_file_perform_op_delegate: cib_query on (null) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: unpack_config: Stop all active resources: false Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: unpack_config: Default stickiness: 0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: apply_system_health: Applying automated node health strategy: migrate-on-red Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-1.test.domain.local has an combined system health of 0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-2.test.domain.local has an combined system health of 0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-3.test.domain.local has an combined system health of 0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-1.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_vrouter [p_vrouter] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_print: vip__management (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter_pub (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_print: vip__public (ocf::fuel:ns_IPaddr2): Stopped Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_conntrackd [p_conntrackd] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: short_print: Slaves: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_haproxy [p_haproxy] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_mysql [p_mysql] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_dns [p_dns] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_heat-engine [p_heat-engine] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-plugin-openvswitch-agent [p_neutron-plugin-openvswitch-agent] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-l3-agent [p_neutron-l3-agent] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-dhcp-agent [p_neutron-dhcp-agent] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-metadata-agent [p_neutron-metadata-agent] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-3.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-2.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_ping_vip__public [ping_vip__public] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_ntp [p_ntp] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__management: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 36 times on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999964 more times on node-2.test.domain.local before being forced off Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 36 times on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999964 more times on node-2.test.domain.local before being forced off Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 36 times on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999964 more times on node-2.test.domain.local before being forced off Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:0 has failed 4 times on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter_pub: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource sysinfo_node-1.test.domain.local are unavailable, unclean or shutting down (node-1.test.domain.local: 0, 1000000) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for sysinfo_node-1.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: Resource sysinfo_node-1.test.domain.local cannot run anywhere Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_vrouter:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_vrouter:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_vrouter:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_vrouter:2 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: Resource p_vrouter:2 cannot run anywhere Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_vrouter instances of a possible 3 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: clone_p_haproxy: Rolling back scores from vip__public Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_haproxy:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_haproxy:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_haproxy:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_haproxy:2 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: Resource p_haproxy:2 cannot run anywhere Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_haproxy instances of a possible 3 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__management Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter_pub Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource vip__public are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for vip__public Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: Resource vip__public cannot run anywhere Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: p_conntrackd:0: Rolling back scores from vip__vrouter Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_conntrackd:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_conntrackd:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: p_conntrackd:2: Rolling back scores from vip__vrouter_pub Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_conntrackd:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_conntrackd:2 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: Resource p_conntrackd:2 cannot run anywhere Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_conntrackd instances of a possible 3 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: master_p_conntrackd: Breaking dependency loop at master_p_conntrackd Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:1 master score: 1000000 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_conntrackd:1 (Slave node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:2 master score: 0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:0 master score: -1000000 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: master_color: master_p_conntrackd: Promoted 1 instances of a possible 1 to master Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_mysql:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_mysql:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_mysql:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_mysql:2 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: Resource p_mysql:2 cannot run anywhere Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_mysql instances of a possible 3 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_rabbitmq-server:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_rabbitmq-server:2 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: Resource p_rabbitmq-server:2 cannot run anywhere Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_rabbitmq-server instances of a possible 3 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:0 master score: 0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_rabbitmq-server:0 (Stopped node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:2 master score: 0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:1 master score: -1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: master_color: master_p_rabbitmq-server: Promoted 1 instances of a possible 1 to master Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:0 with p_vrouter:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_dns:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:1 with p_vrouter:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_dns:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:2 with p_vrouter:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_dns:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_dns:2 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: Resource p_dns:2 cannot run anywhere Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_dns instances of a possible 3 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_heat-engine:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_heat-engine:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_heat-engine:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_heat-engine:2 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: Resource p_heat-engine:2 cannot run anywhere Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_heat-engine instances of a possible 3 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-plugin-openvswitch-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-plugin-openvswitch-agent:2 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-plugin-openvswitch-agent:2 cannot run anywhere Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-plugin-openvswitch-agent instances of a possible 3 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-l3-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-l3-agent:2 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-l3-agent:2 cannot run anywhere Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-l3-agent instances of a possible 3 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-dhcp-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-dhcp-agent:2 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-dhcp-agent:2 cannot run anywhere Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-dhcp-agent instances of a possible 3 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-metadata-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-metadata-agent:2 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-metadata-agent:2 cannot run anywhere Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-metadata-agent instances of a possible 3 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to sysinfo_node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to sysinfo_node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to ping_vip__public:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to ping_vip__public:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource ping_vip__public:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for ping_vip__public:2 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: Resource ping_vip__public:2 cannot run anywhere Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_ping_vip__public instances of a possible 3 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_ntp:0 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_ntp:1 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_ntp:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_ntp:2 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: native_color: Resource p_ntp:2 cannot run anywhere Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_ntp instances of a possible 3 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-plugin-openvswitch-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-l3-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-dhcp-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-metadata-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing sysinfo_node-3.test.domain.local on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing sysinfo_node-2.test.domain.local on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing ping_vip__public:1 on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_ntp:1 on node-3.test.domain.local (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_conntrackd Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:0 on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:1 on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_rabbitmq-server Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:0 on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:1 on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:0 on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:1 on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:0 on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:1 on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:0 on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:1 on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:0 on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:1 on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:0 on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:1 on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (15s) for sysinfo_node-3.test.domain.local on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (15s) for sysinfo_node-2.test.domain.local on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:0 on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:1 on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:0 on node-2.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:1 on node-3.test.domain.local Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_conntrackd:1 (Slave -> Master node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:0 (node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:1 (node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_rabbitmq-server:0 (node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_rabbitmq-server:0 (Stopped -> Master node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_rabbitmq-server:1 (node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_dns:0 (node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_dns:1 (node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_heat-engine:0 (node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_heat-engine:1 (node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:0 (node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:1 (node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:0 (node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:1 (node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:0 (node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:1 (node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:0 (node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:1 (node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start sysinfo_node-3.test.domain.local (node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start sysinfo_node-2.test.domain.local (node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:0 (node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:1 (node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:0 (node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:1 (node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:0 (aka. p_dns) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:0 (aka. p_dns) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:0 (aka. p_heat-engine) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:0 (aka. p_heat-engine) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: debug: cib_file_signoff: Signing out of the CIB Service Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: cib_file_signoff: Wrote CIB to /var/lib/pacemaker/cib/shadow.13192 Feb 17 18:34:58 [13192] node-3.test.domain.local crm_simulate: info: crm_xml_cleanup: Cleaning up memory from libxml2 ocf-mysql-wss: 2016/02/17_18:34:58 INFO: Possible masters: node-2.test.domain.local ocf-mysql-wss: 2016/02/17_18:34:58 INFO: Choosed master: node-2.test.domain.local Feb 17 18:34:58 [13215] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:34:58 [13215] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:34:58 [13215] node-3.test.domain.local crm_node: debug: main: Attempting to process -q command for cluster type: corosync Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-13215-26) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [13215] Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:34:58 [13215] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [13215] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [13215] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7f761349cf10 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7f761349cf10 Feb 17 18:34:58 [13215] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:58 [13215] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-request-6268-13215-26-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-13215-26) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-13215-26) state:2 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7f761349cf10 Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-6268-13215-26-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-6268-13215-26-header Feb 17 18:34:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-6268-13215-26-header Feb 17 18:34:58 [13215] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-response-6268-13215-26-header Feb 17 18:34:58 [13215] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-event-6268-13215-26-header Feb 17 18:34:58 [13215] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:58 [13224] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:34:58 [13224] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:34:58 [13224] node-3.test.domain.local crm_node: debug: main: Attempting to process -l command for cluster type: corosync Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=13224 id=b76b7386-9adb-4562-87d9-5e39c5793ce4 Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-13224-12) Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [13224] Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [13224] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [13224] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [13224] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:34:58 [13224] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-13224-12) Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-13224-12) state:2 Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-13224-12-header Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-13224-12-header Feb 17 18:34:58 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-13224-12-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x167efa0 for uid=0 gid=0 pid=13228 id=5bf78875-0b85-4099-94c5-fba1c946bda2 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-13228-14) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13228] Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: unpack_config: Stop all active resources: false Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: unpack_config: Default stickiness: 0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:0 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13228-14-header Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13228-14-header Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13228-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-13228-14) Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-13228-14) state:2 Feb 17 18:34:58 [13228] node-3.test.domain.local crm_resource: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13228-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13228-14-header Feb 17 18:34:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13228-14-header Feb 17 18:35:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:35:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:35:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:35:03 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:35:03 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:35:03 INFO: haproxy daemon running Feb 17 18:35:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:35:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11983) = 0: Resource temporarily unavailable (11) Feb 17 18:35:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13286) = 0: Resource temporarily unavailable (11) Feb 17 18:35:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13305) = 0: Resource temporarily unavailable (11) Feb 17 18:35:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:13336 - exited with rc=0 Feb 17 18:35:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:13336:stderr [ -- empty -- ] Feb 17 18:35:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:13336:stdout [ -- empty -- ] Feb 17 18:35:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:13336 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:03 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 12fc4935182dd18277a5a2b030371a53 for 0.155.478 (0x1733cf0 0) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header lrmd: 2016/02/17_18:35:03 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 2: /usr/sbin/rabbitmqctl start_app >>/var/log/rabbitmq/startup_log 2>&1 lrmd: 2016/02/17_18:35:03 INFO: p_rabbitmq-server: try_to_start_rmq_app(): start_app failed. lrmd: 2016/02/17_18:35:03 WARNING: p_rabbitmq-server: start_rmq_server_app(): RMQ-server app can't start without Mnesia cleaning. Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:35:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:35:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1be0 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.478 2 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.479 (null) Feb 17 18:35:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=479 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-master-p_rabbitmq-server']: @value=1 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/737, version=0.155.479) Feb 17 18:35:03 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.479 to 0.155.478 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1be0 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:35:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:35:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:35:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_rabbitmq-server'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[3]) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:03 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:35:03 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/41) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:03 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:03 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/41, version=0.155.479) Feb 17 18:35:03 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 41 for master-p_rabbitmq-server=0 passed Feb 17 18:35:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header lrmd: 2016/02/17_18:35:04 INFO: get_status(): app rabbit was not found in command output: [{sasl,"SASL CXC 138 11","2.6"}, {stdlib,"ERTS CXC 138 10","2.6"}, {kernel,"ERTS CXC 138 10","4.1"}] lrmd: 2016/02/17_18:35:04 INFO: p_rabbitmq-server: reset_mnesia(): Execute reset with timeout: 90 lrmd: 2016/02/17_18:35:04 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl reset ocf-ns_IPaddr2: 2016/02/17_18:35:04 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.766ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.794ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.533ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.541ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:35:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:35:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11983) = 0: Interrupted system call (4) Feb 17 18:35:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:13286 - exited with rc=0 Feb 17 18:35:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:13286:stderr [ -- empty -- ] Feb 17 18:35:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:13286:stdout [ -- empty -- ] Feb 17 18:35:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:13286 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:35:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13305) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:35:04 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.769ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.540ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:35:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:35:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11983) = 0: Resource temporarily unavailable (11) Feb 17 18:35:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:13305 - exited with rc=0 Feb 17 18:35:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:13305:stderr [ -- empty -- ] Feb 17 18:35:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:13305:stdout [ -- empty -- ] Feb 17 18:35:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:13305 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:04 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.479 2 Feb 17 18:35:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.480 (null) Feb 17 18:35:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=480 Feb 17 18:35:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_rabbitmq-server']/lrm_rsc_op[@id='p_rabbitmq-server_last_0']: @operation_key=p_rabbitmq-server_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=83:79:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;83:79:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=252, @last-run=1455734088, @last-rc-change Feb 17 18:35:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/343, version=0.155.480) Feb 17 18:35:04 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.480 to 0.155.479 Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:06 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.480 2 Feb 17 18:35:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.481 (null) Feb 17 18:35:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=481 Feb 17 18:35:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_mysql']/lrm_rsc_op[@id='p_mysql_last_0']: @operation_key=p_mysql_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=75:79:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;75:79:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=251, @last-run=1455734088, @last-rc-change=1455734088, @exec-time=18367 Feb 17 18:35:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/344, version=0.155.481) Feb 17 18:35:06 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.481 to 0.155.480 Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-mysql-wss: 2016/02/17_18:35:08 INFO: Waiting for master. 290 seconds left Feb 17 18:35:08 [13759] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:35:08 [13759] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:35:08 [13759] node-3.test.domain.local crm_node: debug: main: Attempting to process -p command for cluster type: corosync Feb 17 18:35:08 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=13759 id=b70c71b1-fb57-4615-958f-1ef50c5f9be0 Feb 17 18:35:08 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-13759-12) Feb 17 18:35:08 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [13759] Feb 17 18:35:08 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:08 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:08 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:08 [13759] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:08 [13759] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:08 [13759] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:08 [13759] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:35:08 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-13759-12) Feb 17 18:35:08 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-13759-12) state:2 Feb 17 18:35:08 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:35:08 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-13759-12-header Feb 17 18:35:08 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-13759-12-header Feb 17 18:35:08 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-13759-12-header Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x167efa0 for uid=0 gid=0 pid=13764 id=353bd417-c89c-4a4e-9ccb-c3dfdf2a1b5d Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-13764-14) Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13764] Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-2.test.domain.local to 2 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='2']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: info: main: Read gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:20770 Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13764-14-header Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13764-14-header Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13764-14-header Feb 17 18:35:08 [13764] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-13764-14) Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-13764-14) state:2 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13764-14-header Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13764-14-header Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13764-14-header ocf-mysql-wss: 2016/02/17_18:35:08 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:20770 ocf-mysql-wss: 2016/02/17_18:35:08 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:20770 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x167efa0 for uid=0 gid=0 pid=13781 id=c6101fae-afbe-4205-9bbf-372f25d22ed1 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-13781-14) Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13781] Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[7]) Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: info: main: Read gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13781-14-header Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13781-14-header Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13781-14-header Feb 17 18:35:08 [13781] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-13781-14) Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-13781-14) state:2 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13781-14-header Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13781-14-header Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13781-14-header ocf-mysql-wss: 2016/02/17_18:35:08 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1 ocf-mysql-wss: 2016/02/17_18:35:08 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x167efa0 for uid=0 gid=0 pid=13801 id=a0d51e0d-a4aa-4a80-8c32-f908eb972e25 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-13801-14) Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13801] Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13801-14-header Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13801-14-header Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13801-14-header Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: validate_with_relaxng: Creating RNG parser context Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-13801-14) Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-13801-14) state:2 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13801-14-header Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13801-14-header Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13801-14-header Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: cib_file_signon: crm_simulate: Opened connection to local file '/var/lib/pacemaker/cib/shadow.13801' Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: cib_file_perform_op_delegate: cib_query on (null) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: unpack_config: Stop all active resources: false Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: unpack_config: Default stickiness: 0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: apply_system_health: Applying automated node health strategy: migrate-on-red Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-1.test.domain.local has an combined system health of 0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-2.test.domain.local has an combined system health of 0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-3.test.domain.local has an combined system health of 0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-1.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_vrouter [p_vrouter] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_print: vip__management (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter_pub (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_print: vip__public (ocf::fuel:ns_IPaddr2): Stopped Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_conntrackd [p_conntrackd] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: short_print: Slaves: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_haproxy [p_haproxy] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_mysql [p_mysql] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local ] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-3.test.domain.local ] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: short_print: Slaves: [ node-2.test.domain.local ] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-3.test.domain.local ] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_dns [p_dns] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_heat-engine [p_heat-engine] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-plugin-openvswitch-agent [p_neutron-plugin-openvswitch-agent] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-l3-agent [p_neutron-l3-agent] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-dhcp-agent [p_neutron-dhcp-agent] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-metadata-agent [p_neutron-metadata-agent] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-3.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-2.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_ping_vip__public [ping_vip__public] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_ntp [p_ntp] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__management: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_mysql:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 36 times on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999964 more times on node-2.test.domain.local before being forced off Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:1 has failed 36 times on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999964 more times on node-2.test.domain.local before being forced off Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 36 times on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999964 more times on node-2.test.domain.local before being forced off Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_rabbitmq-server:0: preferring current location (node=node-2.test.domain.local, weight=100) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:0 has failed 4 times on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter_pub: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource sysinfo_node-1.test.domain.local are unavailable, unclean or shutting down (node-1.test.domain.local: 0, 1000000) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for sysinfo_node-1.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: Resource sysinfo_node-1.test.domain.local cannot run anywhere Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_vrouter:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_vrouter:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_vrouter:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_vrouter:2 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: Resource p_vrouter:2 cannot run anywhere Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_vrouter instances of a possible 3 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: clone_p_haproxy: Rolling back scores from vip__public Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_haproxy:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_haproxy:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_haproxy:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_haproxy:2 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: Resource p_haproxy:2 cannot run anywhere Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_haproxy instances of a possible 3 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__management Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter_pub Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource vip__public are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for vip__public Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: Resource vip__public cannot run anywhere Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: p_conntrackd:0: Rolling back scores from vip__vrouter Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_conntrackd:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_conntrackd:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: p_conntrackd:2: Rolling back scores from vip__vrouter_pub Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_conntrackd:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_conntrackd:2 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: Resource p_conntrackd:2 cannot run anywhere Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_conntrackd instances of a possible 3 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: master_p_conntrackd: Breaking dependency loop at master_p_conntrackd Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:1 master score: 1000000 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_conntrackd:1 (Slave node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:2 master score: 0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:0 master score: -1000000 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: master_color: master_p_conntrackd: Promoted 1 instances of a possible 1 to master Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_mysql:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_mysql:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_mysql:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_mysql:2 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: Resource p_mysql:2 cannot run anywhere Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_mysql instances of a possible 3 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_rabbitmq-server:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_rabbitmq-server:2 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: Resource p_rabbitmq-server:2 cannot run anywhere Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_rabbitmq-server instances of a possible 3 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:0 master score: 1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_rabbitmq-server:0 (Slave node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:1 master score: 0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:2 master score: 0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: master_color: master_p_rabbitmq-server: Promoted 1 instances of a possible 1 to master Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:0 with p_vrouter:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_dns:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:1 with p_vrouter:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_dns:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:2 with p_vrouter:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_dns:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_dns:2 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: Resource p_dns:2 cannot run anywhere Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_dns instances of a possible 3 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_heat-engine:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_heat-engine:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_heat-engine:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_heat-engine:2 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: Resource p_heat-engine:2 cannot run anywhere Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_heat-engine instances of a possible 3 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-plugin-openvswitch-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-plugin-openvswitch-agent:2 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-plugin-openvswitch-agent:2 cannot run anywhere Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-plugin-openvswitch-agent instances of a possible 3 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-l3-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-l3-agent:2 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-l3-agent:2 cannot run anywhere Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-l3-agent instances of a possible 3 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-dhcp-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-dhcp-agent:2 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-dhcp-agent:2 cannot run anywhere Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-dhcp-agent instances of a possible 3 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-metadata-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-metadata-agent:2 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-metadata-agent:2 cannot run anywhere Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-metadata-agent instances of a possible 3 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to sysinfo_node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to sysinfo_node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to ping_vip__public:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to ping_vip__public:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource ping_vip__public:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for ping_vip__public:2 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: Resource ping_vip__public:2 cannot run anywhere Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_ping_vip__public instances of a possible 3 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_ntp:0 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_ntp:1 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_ntp:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_ntp:2 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: native_color: Resource p_ntp:2 cannot run anywhere Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_ntp instances of a possible 3 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-plugin-openvswitch-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-l3-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-dhcp-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-metadata-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing sysinfo_node-3.test.domain.local on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing sysinfo_node-2.test.domain.local on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing ping_vip__public:1 on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_ntp:1 on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_conntrackd Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:0 on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:1 on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_rabbitmq-server Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:0 on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:1 on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:0 on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:1 on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:0 on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:1 on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:0 on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:1 on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:0 on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:1 on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:0 on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:1 on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (15s) for sysinfo_node-3.test.domain.local on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (15s) for sysinfo_node-2.test.domain.local on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:0 on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:1 on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:0 on node-2.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:1 on node-3.test.domain.local Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_conntrackd:1 (Slave -> Master node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:0 (Started node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:1 (node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_rabbitmq-server:0 (Slave -> Master node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_rabbitmq-server:1 (node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_dns:0 (node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_dns:1 (node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_heat-engine:0 (node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_heat-engine:1 (node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:0 (node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:1 (node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:0 (node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:1 (node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:0 (node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:1 (node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:0 (node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:1 (node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start sysinfo_node-3.test.domain.local (node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start sysinfo_node-2.test.domain.local (node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:0 (node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:1 (node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:0 (node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:1 (node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. p_mysql) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. p_mysql) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:0 (aka. p_dns) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:0 (aka. p_dns) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:0 (aka. p_heat-engine) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:0 (aka. p_heat-engine) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:0 (Started node-2.test.domain.local) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: debug: cib_file_signoff: Signing out of the CIB Service Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: cib_file_signoff: Wrote CIB to /var/lib/pacemaker/cib/shadow.13801 Feb 17 18:35:08 [13801] node-3.test.domain.local crm_simulate: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x167efa0 for uid=0 gid=0 pid=13812 id=717d9bae-4646-4c5c-b3ae-8f9667ccd9c1 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-13812-14) Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13812] Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13812-14-header Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13812-14-header Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13812-14-header Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: validate_with_relaxng: Creating RNG parser context Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-13812-14) Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-13812-14) state:2 Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13812-14-header Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13812-14-header Feb 17 18:35:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13812-14-header Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: cib_file_signon: crm_simulate: Opened connection to local file '/var/lib/pacemaker/cib/shadow.13812' Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: cib_file_perform_op_delegate: cib_query on (null) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: unpack_config: Stop all active resources: false Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: unpack_config: Default stickiness: 0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: apply_system_health: Applying automated node health strategy: migrate-on-red Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-1.test.domain.local has an combined system health of 0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-2.test.domain.local has an combined system health of 0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-3.test.domain.local has an combined system health of 0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-1.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_vrouter [p_vrouter] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_print: vip__management (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter_pub (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_print: vip__public (ocf::fuel:ns_IPaddr2): Stopped Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_conntrackd [p_conntrackd] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: short_print: Slaves: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_haproxy [p_haproxy] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_mysql [p_mysql] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local ] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-3.test.domain.local ] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: short_print: Slaves: [ node-2.test.domain.local ] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-3.test.domain.local ] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_dns [p_dns] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_heat-engine [p_heat-engine] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-plugin-openvswitch-agent [p_neutron-plugin-openvswitch-agent] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-l3-agent [p_neutron-l3-agent] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-dhcp-agent [p_neutron-dhcp-agent] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-metadata-agent [p_neutron-metadata-agent] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-3.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-2.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_ping_vip__public [ping_vip__public] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_ntp [p_ntp] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local ] Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__management: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_mysql:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 36 times on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999964 more times on node-2.test.domain.local before being forced off Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:1 has failed 36 times on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999964 more times on node-2.test.domain.local before being forced off Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 36 times on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999964 more times on node-2.test.domain.local before being forced off Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_rabbitmq-server:0: preferring current location (node=node-2.test.domain.local, weight=100) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:0 has failed 4 times on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter_pub: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource sysinfo_node-1.test.domain.local are unavailable, unclean or shutting down (node-1.test.domain.local: 0, 1000000) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for sysinfo_node-1.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: Resource sysinfo_node-1.test.domain.local cannot run anywhere Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_vrouter:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_vrouter:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_vrouter:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_vrouter:2 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: Resource p_vrouter:2 cannot run anywhere Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_vrouter instances of a possible 3 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: clone_p_haproxy: Rolling back scores from vip__public Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_haproxy:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_haproxy:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_haproxy:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_haproxy:2 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: Resource p_haproxy:2 cannot run anywhere Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_haproxy instances of a possible 3 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__management Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter_pub Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource vip__public are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for vip__public Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: Resource vip__public cannot run anywhere Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: p_conntrackd:0: Rolling back scores from vip__vrouter Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_conntrackd:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_conntrackd:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: p_conntrackd:2: Rolling back scores from vip__vrouter_pub Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_conntrackd:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_conntrackd:2 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: Resource p_conntrackd:2 cannot run anywhere Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_conntrackd instances of a possible 3 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: master_p_conntrackd: Breaking dependency loop at master_p_conntrackd Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:1 master score: 1000000 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_conntrackd:1 (Slave node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:2 master score: 0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:0 master score: -1000000 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: master_color: master_p_conntrackd: Promoted 1 instances of a possible 1 to master Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_mysql:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_mysql:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_mysql:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_mysql:2 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: Resource p_mysql:2 cannot run anywhere Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_mysql instances of a possible 3 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_rabbitmq-server:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_rabbitmq-server:2 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: Resource p_rabbitmq-server:2 cannot run anywhere Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_rabbitmq-server instances of a possible 3 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:0 master score: 1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_rabbitmq-server:0 (Slave node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:1 master score: 0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:2 master score: 0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: master_color: master_p_rabbitmq-server: Promoted 1 instances of a possible 1 to master Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:0 with p_vrouter:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_dns:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:1 with p_vrouter:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_dns:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:2 with p_vrouter:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_dns:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_dns:2 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: Resource p_dns:2 cannot run anywhere Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_dns instances of a possible 3 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_heat-engine:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_heat-engine:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_heat-engine:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_heat-engine:2 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: Resource p_heat-engine:2 cannot run anywhere Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_heat-engine instances of a possible 3 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-plugin-openvswitch-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-plugin-openvswitch-agent:2 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-plugin-openvswitch-agent:2 cannot run anywhere Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-plugin-openvswitch-agent instances of a possible 3 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-l3-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-l3-agent:2 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-l3-agent:2 cannot run anywhere Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-l3-agent instances of a possible 3 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-dhcp-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-dhcp-agent:2 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-dhcp-agent:2 cannot run anywhere Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-dhcp-agent instances of a possible 3 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-metadata-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-metadata-agent:2 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-metadata-agent:2 cannot run anywhere Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-metadata-agent instances of a possible 3 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to sysinfo_node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to sysinfo_node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to ping_vip__public:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to ping_vip__public:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource ping_vip__public:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for ping_vip__public:2 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: Resource ping_vip__public:2 cannot run anywhere Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_ping_vip__public instances of a possible 3 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_ntp:0 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_ntp:1 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_ntp:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_ntp:2 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: native_color: Resource p_ntp:2 cannot run anywhere Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_ntp instances of a possible 3 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-plugin-openvswitch-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-l3-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-dhcp-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_neutron-metadata-agent:1 on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing sysinfo_node-3.test.domain.local on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing sysinfo_node-2.test.domain.local on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing ping_vip__public:1 on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: native_create_probe: Probing p_ntp:1 on node-3.test.domain.local (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_conntrackd Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:0 on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:1 on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_rabbitmq-server Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:0 on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:1 on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:0 on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:1 on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:0 on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:1 on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:0 on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:1 on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:0 on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:1 on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:0 on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:1 on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (15s) for sysinfo_node-3.test.domain.local on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (15s) for sysinfo_node-2.test.domain.local on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:0 on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:1 on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:0 on node-2.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:1 on node-3.test.domain.local Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_conntrackd:1 (Slave -> Master node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:0 (Started node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:1 (node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_rabbitmq-server:0 (Slave -> Master node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_rabbitmq-server:1 (node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_dns:0 (node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_dns:1 (node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_heat-engine:0 (node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_heat-engine:1 (node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:0 (node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:1 (node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:0 (node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:1 (node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:0 (node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:1 (node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:0 (node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:1 (node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start sysinfo_node-3.test.domain.local (node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start sysinfo_node-2.test.domain.local (node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:0 (node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:1 (node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:0 (node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:1 (node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. p_conntrackd) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. p_mysql) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. p_mysql) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:0 (aka. p_dns) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:0 (aka. p_dns) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:0 (aka. p_heat-engine) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:0 (aka. p_heat-engine) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:0 (Started node-2.test.domain.local) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: debug: cib_file_signoff: Signing out of the CIB Service Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: cib_file_signoff: Wrote CIB to /var/lib/pacemaker/cib/shadow.13812 Feb 17 18:35:08 [13812] node-3.test.domain.local crm_simulate: info: crm_xml_cleanup: Cleaning up memory from libxml2 ocf-mysql-wss: 2016/02/17_18:35:08 INFO: Possible masters: node-2.test.domain.local ocf-mysql-wss: 2016/02/17_18:35:08 INFO: Choosed master: node-2.test.domain.local Feb 17 18:35:08 [13835] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:35:08 [13835] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:35:08 [13835] node-3.test.domain.local crm_node: debug: main: Attempting to process -q command for cluster type: corosync Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-13835-26) Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [13835] Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:08 [13835] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:08 [13835] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:08 [13835] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7f761349cf10 Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7f761349cf10 Feb 17 18:35:08 [13835] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:08 [13835] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-request-6268-13835-26-header Feb 17 18:35:08 [13835] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-response-6268-13835-26-header Feb 17 18:35:08 [13835] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-event-6268-13835-26-header Feb 17 18:35:08 [13835] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-13835-26) Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-13835-26) state:2 Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7f761349cf10 Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-6268-13835-26-header Feb 17 18:35:09 [13844] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:35:09 [13844] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:35:09 [13844] node-3.test.domain.local crm_node: debug: main: Attempting to process -l command for cluster type: corosync Feb 17 18:35:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-6268-13835-26-header Feb 17 18:35:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-6268-13835-26-header Feb 17 18:35:09 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=13844 id=0dde1c29-a31e-4b1a-8f97-8f3e6d787edd Feb 17 18:35:09 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-13844-12) Feb 17 18:35:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [13844] Feb 17 18:35:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:09 [13844] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:09 [13844] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:09 [13844] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:09 [13844] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:35:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-13844-12) Feb 17 18:35:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-13844-12) state:2 Feb 17 18:35:09 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:35:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-13844-12-header Feb 17 18:35:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-13844-12-header Feb 17 18:35:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-13844-12-header Feb 17 18:35:09 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x167efa0 for uid=0 gid=0 pid=13848 id=186de914-ce98-4987-a5f8-90bfacf2a41f Feb 17 18:35:09 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-13848-14) Feb 17 18:35:09 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13848] Feb 17 18:35:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:35:09 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: unpack_config: Stop all active resources: false Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: unpack_config: Default stickiness: 0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:1 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:0 Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13848-14-header Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13848-14-header Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13848-14-header Feb 17 18:35:09 [13848] node-3.test.domain.local crm_resource: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:35:09 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-13848-14) Feb 17 18:35:09 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-13848-14) state:2 Feb 17 18:35:09 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:35:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-13848-14-header Feb 17 18:35:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-13848-14-header Feb 17 18:35:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-13848-14-header ocf-mysql-wss: 2016/02/17_18:35:09 INFO: My neighbour is Primary Component ocf-mysql-wss: 2016/02/17_18:35:09 INFO: Starting MySQL Feb 17 18:35:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:35:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:11 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:11 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:11 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: b125bbb7b066a213289e6ec8190408a1 for 0.155.481 (0x1733cf0 0) Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:11 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_IPaddr2: 2016/02/17_18:35:12 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.594ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.662ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.548ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.560ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:35:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:35:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11983) = 0: Interrupted system call (4) Feb 17 18:35:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14582 - exited with rc=0 Feb 17 18:35:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14582:stderr [ -- empty -- ] Feb 17 18:35:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14582:stdout [ -- empty -- ] Feb 17 18:35:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:14582 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:35:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14633) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:35:12 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 1.597ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.531ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.514ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:35:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:35:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11983) = 0: Resource temporarily unavailable (11) Feb 17 18:35:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14633 - exited with rc=0 Feb 17 18:35:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14633:stderr [ -- empty -- ] Feb 17 18:35:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14633:stdout [ -- empty -- ] Feb 17 18:35:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:14633 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:35:14 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:35:14 INFO: MySQL is not running lrmd: 2016/02/17_18:35:14 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 2: /usr/sbin/rabbitmqctl reset lrmd: 2016/02/17_18:35:14 INFO: p_rabbitmq-server: reset_mnesia(): Execute force_reset with timeout: 90 lrmd: 2016/02/17_18:35:14 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl force_reset lrmd: 2016/02/17_18:35:14 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl force_reset lrmd: 2016/02/17_18:35:15 INFO: p_rabbitmq-server: try_to_start_rmq_app(): begin. lrmd: 2016/02/17_18:35:15 INFO: p_rabbitmq-server: try_to_start_rmq_app(): Execute start_app with timeout: 90 lrmd: 2016/02/17_18:35:15 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl start_app >>/var/log/rabbitmq/startup_log 2>&1 lrmd: 2016/02/17_18:35:16 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl start_app >>/var/log/rabbitmq/startup_log 2>&1 lrmd: 2016/02/17_18:35:16 INFO: p_rabbitmq-server: try_to_start_rmq_app(): start_app was successful. lrmd: 2016/02/17_18:35:16 INFO: p_rabbitmq-server: try_to_start_rmq_app(): waiting for start to finish with timeout: 90 lrmd: 2016/02/17_18:35:16 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl wait /var/run/rabbitmq/p_pid lrmd: 2016/02/17_18:35:17 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl wait /var/run/rabbitmq/p_pid lrmd: 2016/02/17_18:35:17 INFO: p_rabbitmq-server: try_to_start_rmq_app(): start plugins. Feb 17 18:35:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:35:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 lrmd: 2016/02/17_18:35:17 INFO: p_rabbitmq-server: try_to_start_rmq_app(): Starting plugins: [rabbitmq_management,rabbitmq_management_agent,rabbitmq_web_dispatch, webmachine,mochiweb,amqp_client] lrmd: 2016/02/17_18:35:18 INFO: p_rabbitmq-server: start_rmq_server_app(): Execute stop_app with timeout: 90 lrmd: 2016/02/17_18:35:18 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl stop_app 2>&1 >> "/var/log/rabbitmq/shutdown_log" lrmd: 2016/02/17_18:35:18 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl stop_app 2>&1 >> "/var/log/rabbitmq/shutdown_log" lrmd: 2016/02/17_18:35:18 INFO: get_status(): app rabbit was not found in command output: [{xmerl,"XML parser","1.3.8"}, {inets,"INETS CXC 138 49","6.0.1"}, {sasl,"SASL CXC 138 11","2.6"}, {stdlib,"ERTS CXC 138 10","2.6"}, {kernel,"ERTS CXC 138 10","4.1"}] lrmd: 2016/02/17_18:35:18 INFO: p_rabbitmq-server: start_rmq_server_app(): RMQ-server app stopped succesfully. lrmd: 2016/02/17_18:35:18 INFO: p_rabbitmq-server: start_rmq_server_app(): RMQ-server app Mnesia cleaned successfully. lrmd: 2016/02/17_18:35:18 INFO: p_rabbitmq-server: master_score(): Updating master score attribute with 1 Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-15827-26) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [15827] Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761349cf10, cpd=0x7f761349d7f4 Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f761349cf10 Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-6268-15827-26-header Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-6268-15827-26-header Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-6268-15827-26-header Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-15827-26) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-15827-26) state:2 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-15827-26-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-15827-26-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-15827-26-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-15827-26) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [15827] Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-15827-26-header Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-15827-26-header Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-15827-26-header Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:35:18 [15827] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-15827-26) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-15827-26) state:2 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-15827-26-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-15827-26-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-15827-26-header Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x167efa0 for uid=0 gid=0 pid=15829 id=74f41aaa-aed8-42b7-8e84-4717f1843300 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-15829-14) Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [15829] Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=15829 id=ba390432-4241-4601-a1d0-b370f3539969 Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-15829-11) Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15829] Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: master-p_rabbitmq-server=1 Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1, Current: 0, Stored: 0 Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of master-p_rabbitmq-server is 1 Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_rabbitmq-server (1) Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_rabbitmq-server'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[3]) Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:35:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/43) Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: master-p_rabbitmq-server=1 for node-3.test.domain.local Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: info: main: Update master-p_rabbitmq-server=1 sent via attrd Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-15829-14-header Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-15829-14-header Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-15829-14-header Feb 17 18:35:18 [15829] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-15829-14) Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-15829-14) state:2 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-15829-14-header Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-15829-14-header Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-15829-14-header Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 43: master-p_rabbitmq-server=1 Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-15829-11) Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-15829-11) state:2 Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-15829-11-header Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-15829-11-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-15829-11-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.481 2 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.482 (null) Feb 17 18:35:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=482 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-master-p_rabbitmq-server']: @value=1 Feb 17 18:35:18 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.482 to 0.155.481 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/43, version=0.155.482) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:18 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 43 for master-p_rabbitmq-server=1 passed Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header lrmd: 2016/02/17_18:35:18 INFO: p_rabbitmq-server: start_rmq_server_app(): end. Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/739, version=0.155.482) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header lrmd: 2016/02/17_18:35:19 INFO: p_rabbitmq-server: start_rmq_server_app(): unblocked access to RMQ port lrmd: 2016/02/17_18:35:19 INFO: p_rabbitmq-server: start: RMQ prepared for start succesfully. lrmd: 2016/02/17_18:35:19 INFO: p_rabbitmq-server: start: action end. Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_start_0:11983 - exited with rc=0 Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:11983:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:11983:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:11983:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:11983:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:11983:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:11983:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:11983:stderr [ Error: {cannot_start_mnesia,{killed,{mnesia_sup,start,[normal,[]]}}} ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_start_0:11983:stdout [ Resetting node 'rabbit@messaging-node-3' ... ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_start_0:11983:stdout [ Forcefully resetting node 'rabbit@messaging-node-3' ... ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_start_0:11983:stdout [ Waiting for 'rabbit@messaging-node-3' ... ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_start_0:11983:stdout [ pid is 12355 ... ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_start_0:11983:stdout [ false ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_rabbitmq-server action:start call_id:69 pid:11983 exit-code:0 exec-time:30606ms queue-time:0ms Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_rabbitmq-server after start op complete (interval=0) Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15361) = 0: Resource temporarily unavailable (11) Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15364) = 0: Resource temporarily unavailable (11) Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-15867-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [15867] Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761349cf10, cpd=0x7f761349d7f4 Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f761349cf10 Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-6268-15867-26-header Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-6268-15867-26-header Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-6268-15867-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-15867-26) Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-15867-26) state:2 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-15867-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-15867-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-15867-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-15867-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [15867] Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-15867-26-header Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-15867-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-15867-26) Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-15867-26-header Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-15867-26) state:2 Feb 17 18:35:19 [15867] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-15867-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-15867-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-15867-26-header Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed rabbitmq-server_meta-data_0 process 15851 exited with rc=0 Feb 17 18:35:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/31) Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_rabbitmq-server_start_0: ok (node=node-3.test.domain.local, call=69, rc=0, cib-update=31, confirmed=true) Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_rabbitmq-server_start_0:69 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\nError: {cannot_start_mnesia,{killed,{mnesia_sup,start,[normal,[]]}}}\n ] Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_rabbitmq-server' with start op Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.482 2 Feb 17 18:35:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.483 (null) Feb 17 18:35:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=483 Feb 17 18:35:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_rabbitmq-server']/lrm_rsc_op[@id='p_rabbitmq-server_last_0']: @operation_key=p_rabbitmq-server_start_0, @operation=start, @transition-key=86:79:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;86:79:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=69, @rc-code=0, @last-run=1455734088, @last-rc-change=1455734088, @exec-time=306 Feb 17 18:35:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/31, version=0.155.483) Feb 17 18:35:19 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.483 to 0.155.482 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=216:79:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_rabbitmq-server_notify_0 Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=70, reply=1, notify=0, exit=4202464 Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_rabbitmq-server action:notify call_id:70 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-15895-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [15895] Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761349cf10, cpd=0x7f761349d7f4 Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f761349cf10 Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-6268-15895-26-header Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-6268-15895-26-header Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-6268-15895-26-header Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-15895-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-15895-26) state:2 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-15895-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-15895-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-15895-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-15895-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [15895] Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-15895-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-15895-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-15895-26) state:2 Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-15895-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-15895-26-header Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-15895-26-header Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:35:19 [15895] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-15895-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-15895-26-header lrmd: 2016/02/17_18:35:19 INFO: p_rabbitmq-server: notify: post-start begin. lrmd: 2016/02/17_18:35:19 INFO: p_rabbitmq-server: my_host(): hostlist is: node-2.test.domain.local node-3.test.domain.local lrmd: 2016/02/17_18:35:19 DEBUG: p_rabbitmq-server: my_host(): comparing 'messaging-node-3' with 'messaging-node-2' lrmd: 2016/02/17_18:35:19 DEBUG: p_rabbitmq-server: my_host(): comparing 'messaging-node-3' with 'messaging-node-3' lrmd: 2016/02/17_18:35:19 WARNING: p_rabbitmq-server: notify: there are no nodes to join to reported on post-start. Nothing to do. lrmd: 2016/02/17_18:35:19 INFO: p_rabbitmq-server: notify: post-start end. Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15361) = 0: Resource temporarily unavailable (11) Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15364) = 0: Resource temporarily unavailable (11) Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_notify_0:15879 - exited with rc=0 Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:15879:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:15879:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:15879:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:15879:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:15879:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:15879:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_notify_0:15879:stdout [ -- empty -- ] Feb 17 18:35:19 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_rabbitmq-server action:notify call_id:70 pid:15879 exit-code:0 exec-time:122ms queue-time:0ms Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_rabbitmq-server after notify op complete (interval=0) Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_rabbitmq-server_notify_0 from 216:79:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455734119-11 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_rabbitmq-server_notify_0: ok (node=node-3.test.domain.local, call=70, rc=0, cib-update=0, confirmed=true) Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_rabbitmq-server_notify_0:70 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n ] Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:35:19 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:35:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header ocf-ns_IPaddr2: 2016/02/17_18:35:20 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.655ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.686ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.533ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.527ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:35:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:35:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:15361 - exited with rc=0 Feb 17 18:35:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:15361:stderr [ -- empty -- ] Feb 17 18:35:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:15361:stdout [ -- empty -- ] Feb 17 18:35:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:15361 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:35:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15364) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:35:20 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.618ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.538ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.522ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:35:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:35:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:15364 - exited with rc=0 Feb 17 18:35:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:15364:stderr [ -- empty -- ] Feb 17 18:35:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:15364:stdout [ -- empty -- ] Feb 17 18:35:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:15364 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:35:22 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:35:22 INFO: MySQL is not running Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.003667 (11 ticks in 30s) Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 1.830000 (full: 1.83 0.77 0.28 2/499 16086) Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: throttle_timer_cb: New throttle mode: 0000 (was 0001) Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: info: throttle_send_command: Updated throttle state to 0000 Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:35:23 [6707] node-3.test.domain.local crmd: debug: throttle_update: Host node-3.test.domain.local supports a maximum of 4 jobs and throttle mode 0000. New job limit is 4 Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:35:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:24 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:35:24 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: f68c9476b08aa598370ae9d576450df5 for 0.155.483 (0x1733cf0 0) Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:35:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:35:24 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:35:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:35:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:35:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:35:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:35:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:35:27 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:35:27 INFO: net.ipv4.ip_forward = 1 Feb 17 18:35:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:35:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(16206) = 0: Resource temporarily unavailable (11) Feb 17 18:35:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(16220) = 0: Resource temporarily unavailable (11) Feb 17 18:35:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:16375 - exited with rc=0 Feb 17 18:35:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:16375:stderr [ -- empty -- ] Feb 17 18:35:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:16375:stdout [ -- empty -- ] Feb 17 18:35:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:16375 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:35:28 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.798ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.830ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.529ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.557ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:35:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:35:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:16206 - exited with rc=0 Feb 17 18:35:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:16206:stderr [ -- empty -- ] Feb 17 18:35:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:16206:stdout [ -- empty -- ] Feb 17 18:35:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:16206 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:35:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(16220) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:35:28 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.870ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.535ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:35:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:35:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:16220 - exited with rc=0 Feb 17 18:35:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:16220:stderr [ -- empty -- ] Feb 17 18:35:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:16220:stdout [ -- empty -- ] Feb 17 18:35:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:16220 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:35:30 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:35:30 INFO: MySQL is not running Feb 17 18:35:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:35:33 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:35:33 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:35:33 INFO: haproxy daemon running Feb 17 18:35:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:35:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:16744 - exited with rc=0 Feb 17 18:35:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:16744:stderr [ -- empty -- ] Feb 17 18:35:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:16744:stdout [ -- empty -- ] Feb 17 18:35:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:16744 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:35:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:35:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:35:36 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.637ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.666ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.530ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.542ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:35:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:35:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:16779 - exited with rc=0 Feb 17 18:35:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:16779:stderr [ -- empty -- ] Feb 17 18:35:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:16779:stdout [ -- empty -- ] Feb 17 18:35:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:16779 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:35:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(16798) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:35:36 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.649ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.540ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:35:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:35:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:16798 - exited with rc=0 Feb 17 18:35:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:16798:stderr [ -- empty -- ] Feb 17 18:35:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:16798:stdout [ -- empty -- ] Feb 17 18:35:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:16798 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:35:38 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:35:38 INFO: MySQL is not running Feb 17 18:35:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:35:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:35:44 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.654ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.686ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.542ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.543ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:35:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:35:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17064 - exited with rc=0 Feb 17 18:35:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17064:stderr [ -- empty -- ] Feb 17 18:35:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17064:stdout [ -- empty -- ] Feb 17 18:35:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:17064 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:35:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17091) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:35:44 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.626ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.532ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:35:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:35:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17091 - exited with rc=0 Feb 17 18:35:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17091:stderr [ -- empty -- ] Feb 17 18:35:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17091:stdout [ -- empty -- ] Feb 17 18:35:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:17091 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:35:46 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:35:46 INFO: MySQL is not running Feb 17 18:35:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:35:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:35:52 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.791ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.822ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.541ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.539ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:35:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:35:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17357 - exited with rc=0 Feb 17 18:35:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17357:stderr [ -- empty -- ] Feb 17 18:35:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17357:stdout [ -- empty -- ] Feb 17 18:35:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:17357 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:35:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17378) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:35:52 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.754ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.526ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.531ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:35:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:35:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17378 - exited with rc=0 Feb 17 18:35:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17378:stderr [ -- empty -- ] Feb 17 18:35:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17378:stdout [ -- empty -- ] Feb 17 18:35:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:17378 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:35:53 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.000333 (1 ticks in 30s) Feb 17 18:35:53 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 1.350000 (full: 1.35 0.76 0.29 2/485 17443) Feb 17 18:35:53 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 ocf-mysql-wss: 2016/02/17_18:35:54 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:35:54 INFO: MySQL is not running Feb 17 18:35:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:35:57 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:35:57 INFO: net.ipv4.ip_forward = 1 Feb 17 18:35:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:35:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:17563 - exited with rc=0 Feb 17 18:35:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:17563:stderr [ -- empty -- ] Feb 17 18:35:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:17563:stdout [ -- empty -- ] Feb 17 18:35:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:17563 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:35:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:35:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:36:00 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.776ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.798ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.540ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.529ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:36:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:36:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17582 - exited with rc=0 Feb 17 18:36:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17582:stderr [ -- empty -- ] Feb 17 18:36:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17582:stdout [ -- empty -- ] Feb 17 18:36:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:17582 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:36:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17601) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:36:01 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.888ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.543ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:36:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:36:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17601 - exited with rc=0 Feb 17 18:36:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17601:stderr [ -- empty -- ] Feb 17 18:36:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17601:stdout [ -- empty -- ] Feb 17 18:36:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:17601 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:36:02 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:36:02 INFO: MySQL is not running Feb 17 18:36:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:36:03 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:36:03 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:36:03 INFO: haproxy daemon running Feb 17 18:36:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:36:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:17777 - exited with rc=0 Feb 17 18:36:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:17777:stderr [ -- empty -- ] Feb 17 18:36:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:17777:stdout [ -- empty -- ] Feb 17 18:36:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:17777 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:36:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:36:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:36:08 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.679ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.707ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.541ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.531ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:36:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:36:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17845 - exited with rc=0 Feb 17 18:36:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17845:stderr [ -- empty -- ] Feb 17 18:36:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17845:stdout [ -- empty -- ] Feb 17 18:36:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:17845 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:36:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17864) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:36:09 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.901ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.534ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:36:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:36:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17864 - exited with rc=0 Feb 17 18:36:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17864:stderr [ -- empty -- ] Feb 17 18:36:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17864:stdout [ -- empty -- ] Feb 17 18:36:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:17864 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:36:10 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:36:10 INFO: MySQL is not running Feb 17 18:36:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:36:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:36:17 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.775ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.807ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.543ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.540ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:36:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:36:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:18065 - exited with rc=0 Feb 17 18:36:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:18065:stderr [ -- empty -- ] Feb 17 18:36:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:18065:stdout [ -- empty -- ] Feb 17 18:36:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:18065 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:36:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(18084) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:36:17 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.807ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.527ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.532ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:36:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:36:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:18084 - exited with rc=0 Feb 17 18:36:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:18084:stderr [ -- empty -- ] Feb 17 18:36:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:18084:stdout [ -- empty -- ] Feb 17 18:36:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:18084 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:36:18 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:36:18 INFO: MySQL is not running Feb 17 18:36:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:36:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:36:23 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.000000 (0 ticks in 30s) Feb 17 18:36:23 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.820000 (full: 0.82 0.68 0.29 1/480 18662) Feb 17 18:36:23 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 ocf-ns_IPaddr2: 2016/02/17_18:36:25 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.625ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.650ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.528ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.530ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:36:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:36:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:18567 - exited with rc=0 Feb 17 18:36:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:18567:stderr [ -- empty -- ] Feb 17 18:36:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:18567:stdout [ -- empty -- ] Feb 17 18:36:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:18567 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:36:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(18586) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:36:25 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.854ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.545ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.540ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:36:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:36:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:18586 - exited with rc=0 Feb 17 18:36:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:18586:stderr [ -- empty -- ] Feb 17 18:36:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:18586:stdout [ -- empty -- ] Feb 17 18:36:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:18586 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:36:26 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:36:26 INFO: MySQL is not running Feb 17 18:36:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:36:27 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:36:27 INFO: net.ipv4.ip_forward = 1 Feb 17 18:36:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:36:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:18953 - exited with rc=0 Feb 17 18:36:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:18953:stderr [ -- empty -- ] Feb 17 18:36:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:18953:stdout [ -- empty -- ] Feb 17 18:36:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:18953 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:36:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:36:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:36:33 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.775ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.809ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.550ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.543ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:19267 - exited with rc=0 Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:19267:stderr [ -- empty -- ] Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:19267:stdout [ -- empty -- ] Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:19267 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(19286) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:36:33 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.673ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.539ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:19286 - exited with rc=0 Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:19286:stderr [ -- empty -- ] Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:19286:stdout [ -- empty -- ] Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:19286 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:36:33 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:36:33 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:36:33 INFO: haproxy daemon running Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:19623 - exited with rc=0 Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:19623:stderr [ -- empty -- ] Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:19623:stdout [ -- empty -- ] Feb 17 18:36:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:19623 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:36:34 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:36:34 INFO: MySQL is not running Feb 17 18:36:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:36:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:36:41 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.770ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.794ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.539ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.538ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:36:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:36:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:19962 - exited with rc=0 Feb 17 18:36:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:19962:stderr [ -- empty -- ] Feb 17 18:36:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:19962:stdout [ -- empty -- ] Feb 17 18:36:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:19962 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:36:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(19981) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:36:41 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.665ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.540ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.540ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:36:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:36:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:19981 - exited with rc=0 Feb 17 18:36:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:19981:stderr [ -- empty -- ] Feb 17 18:36:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:19981:stdout [ -- empty -- ] Feb 17 18:36:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:19981 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:36:42 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:36:42 INFO: MySQL is not running Feb 17 18:36:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:36:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:36:49 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.638ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.665ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.526ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.560ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:36:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:36:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:20433 - exited with rc=0 Feb 17 18:36:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:20433:stderr [ -- empty -- ] Feb 17 18:36:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:20433:stdout [ -- empty -- ] Feb 17 18:36:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:20433 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:36:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(20452) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:36:49 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.794ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.532ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.539ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:36:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:36:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:20452 - exited with rc=0 Feb 17 18:36:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:20452:stderr [ -- empty -- ] Feb 17 18:36:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:20452:stdout [ -- empty -- ] Feb 17 18:36:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:20452 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:36:50 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:36:50 INFO: MySQL is not running Feb 17 18:36:53 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.000000 (0 ticks in 30s) Feb 17 18:36:53 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.660000 (full: 0.66 0.67 0.29 1/464 20888) Feb 17 18:36:53 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:36:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:36:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:36:57 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.833ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.854ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.540ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.542ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:20961 - exited with rc=0 Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:20961:stderr [ -- empty -- ] Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:20961:stdout [ -- empty -- ] Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:20961 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(20980) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:36:57 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.785ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.533ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:20980 - exited with rc=0 Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:20980:stderr [ -- empty -- ] Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:20980:stdout [ -- empty -- ] Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:20980 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:36:57 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:36:57 INFO: net.ipv4.ip_forward = 1 Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:21217 - exited with rc=0 Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:21217:stderr [ -- empty -- ] Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:21217:stdout [ -- empty -- ] Feb 17 18:36:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:21217 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:36:58 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:36:58 INFO: MySQL is not running Feb 17 18:37:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:37:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:37:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:37:03 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:37:03 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:37:03 INFO: haproxy daemon running Feb 17 18:37:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:37:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21519) = 0: Resource temporarily unavailable (11) Feb 17 18:37:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21538) = 0: Resource temporarily unavailable (11) Feb 17 18:37:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:21642 - exited with rc=0 Feb 17 18:37:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:21642:stderr [ -- empty -- ] Feb 17 18:37:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:21642:stdout [ -- empty -- ] Feb 17 18:37:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:21642 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:37:05 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.806ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.836ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.532ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.544ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:37:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:37:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:21519 - exited with rc=0 Feb 17 18:37:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:21519:stderr [ -- empty -- ] Feb 17 18:37:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:21519:stdout [ -- empty -- ] Feb 17 18:37:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:21519 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:37:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21538) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:37:05 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.819ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.538ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.532ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:37:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:37:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:21538 - exited with rc=0 Feb 17 18:37:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:21538:stderr [ -- empty -- ] Feb 17 18:37:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:21538:stdout [ -- empty -- ] Feb 17 18:37:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:21538 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:37:06 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:37:06 INFO: MySQL is not running Feb 17 18:37:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:37:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:37:13 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.785ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.809ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.540ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.526ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:37:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:37:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:22288 - exited with rc=0 Feb 17 18:37:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:22288:stderr [ -- empty -- ] Feb 17 18:37:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:22288:stdout [ -- empty -- ] Feb 17 18:37:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:22288 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:37:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(22310) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:37:13 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.652ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.526ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.534ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:37:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:37:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:22310 - exited with rc=0 Feb 17 18:37:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:22310:stderr [ -- empty -- ] Feb 17 18:37:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:22310:stdout [ -- empty -- ] Feb 17 18:37:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:22310 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:37:14 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:37:14 INFO: MySQL is not running Feb 17 18:37:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:37:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:37:21 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.635ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.662ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.543ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.537ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:37:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:37:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:22832 - exited with rc=0 Feb 17 18:37:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:22832:stderr [ -- empty -- ] Feb 17 18:37:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:22832:stdout [ -- empty -- ] Feb 17 18:37:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:22832 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:37:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(22851) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:37:21 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.654ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.567ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:37:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:37:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:22851 - exited with rc=0 Feb 17 18:37:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:22851:stderr [ -- empty -- ] Feb 17 18:37:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:22851:stdout [ -- empty -- ] Feb 17 18:37:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:22851 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:37:22 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:37:22 INFO: MySQL is not running Feb 17 18:37:23 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.000000 (0 ticks in 30s) Feb 17 18:37:23 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.610000 (full: 0.61 0.65 0.30 2/462 23151) Feb 17 18:37:23 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:37:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:37:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:37:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:37:27 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:37:27 INFO: net.ipv4.ip_forward = 1 Feb 17 18:37:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:37:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(23354) = 0: Resource temporarily unavailable (11) Feb 17 18:37:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(23373) = 0: Resource temporarily unavailable (11) Feb 17 18:37:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:23454 - exited with rc=0 Feb 17 18:37:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:23454:stderr [ -- empty -- ] Feb 17 18:37:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:23454:stdout [ -- empty -- ] Feb 17 18:37:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:23454 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:37:29 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.816ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.851ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.523ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.544ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:37:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:37:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:23354 - exited with rc=0 Feb 17 18:37:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:23354:stderr [ -- empty -- ] Feb 17 18:37:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:23354:stdout [ -- empty -- ] Feb 17 18:37:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:23354 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:37:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(23373) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:37:29 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.733ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.527ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:37:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:37:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:23373 - exited with rc=0 Feb 17 18:37:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:23373:stderr [ -- empty -- ] Feb 17 18:37:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:23373:stdout [ -- empty -- ] Feb 17 18:37:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:23373 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:37:30 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:37:30 INFO: MySQL is not running Feb 17 18:37:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:37:33 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:37:33 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:37:33 INFO: haproxy daemon running Feb 17 18:37:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:37:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:23826 - exited with rc=0 Feb 17 18:37:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:23826:stderr [ -- empty -- ] Feb 17 18:37:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:23826:stdout [ -- empty -- ] Feb 17 18:37:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:23826 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:37:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:37:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:37:37 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.706ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.729ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.528ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.536ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:37:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:37:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:23858 - exited with rc=0 Feb 17 18:37:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:23858:stderr [ -- empty -- ] Feb 17 18:37:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:23858:stdout [ -- empty -- ] Feb 17 18:37:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:23858 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:37:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(23904) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:37:37 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.809ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.540ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.527ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:37:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:37:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:23904 - exited with rc=0 Feb 17 18:37:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:23904:stderr [ -- empty -- ] Feb 17 18:37:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:23904:stdout [ -- empty -- ] Feb 17 18:37:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:23904 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:37:38 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:37:38 INFO: MySQL is not running Feb 17 18:37:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:37:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:37:45 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.760ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.797ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.536ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.537ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:37:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:37:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:24370 - exited with rc=0 Feb 17 18:37:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:24370:stderr [ -- empty -- ] Feb 17 18:37:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:24370:stdout [ -- empty -- ] Feb 17 18:37:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:24370 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:37:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(24389) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:37:45 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.981ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.529ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.544ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:37:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:37:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:24389 - exited with rc=0 Feb 17 18:37:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:24389:stderr [ -- empty -- ] Feb 17 18:37:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:24389:stdout [ -- empty -- ] Feb 17 18:37:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:24389 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:37:46 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:37:46 INFO: MySQL is not running Feb 17 18:37:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:37:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:37:53 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.000000 (0 ticks in 30s) Feb 17 18:37:53 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.560000 (full: 0.56 0.64 0.31 2/464 25049) Feb 17 18:37:53 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 ocf-ns_IPaddr2: 2016/02/17_18:37:53 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.680ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.734ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.533ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.525ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:37:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:37:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:24883 - exited with rc=0 Feb 17 18:37:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:24883:stderr [ -- empty -- ] Feb 17 18:37:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:24883:stdout [ -- empty -- ] Feb 17 18:37:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:24883 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:37:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(24902) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:37:53 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.779ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.548ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.526ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:37:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:37:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:24902 - exited with rc=0 Feb 17 18:37:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:24902:stderr [ -- empty -- ] Feb 17 18:37:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:24902:stdout [ -- empty -- ] Feb 17 18:37:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:24902 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:37:54 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:37:54 INFO: MySQL is not running Feb 17 18:37:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:37:57 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:37:57 INFO: net.ipv4.ip_forward = 1 Feb 17 18:37:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:37:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:25364 - exited with rc=0 Feb 17 18:37:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:25364:stderr [ -- empty -- ] Feb 17 18:37:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:25364:stdout [ -- empty -- ] Feb 17 18:37:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:25364 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:37:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:37:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:38:01 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.761ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.786ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.530ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.537ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:38:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:38:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:25391 - exited with rc=0 Feb 17 18:38:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:25391:stderr [ -- empty -- ] Feb 17 18:38:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:25391:stdout [ -- empty -- ] Feb 17 18:38:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:25391 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:38:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(25437) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:38:01 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.851ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.539ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:38:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:38:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:25437 - exited with rc=0 Feb 17 18:38:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:25437:stderr [ -- empty -- ] Feb 17 18:38:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:25437:stdout [ -- empty -- ] Feb 17 18:38:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:25437 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:38:02 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:38:02 INFO: MySQL is not running Feb 17 18:38:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:38:03 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:38:03 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:38:03 INFO: haproxy daemon running Feb 17 18:38:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:38:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:25773 - exited with rc=0 Feb 17 18:38:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:25773:stderr [ -- empty -- ] Feb 17 18:38:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:25773:stdout [ -- empty -- ] Feb 17 18:38:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:25773 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:38:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:38:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:38:09 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.721ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.743ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.532ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.544ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:38:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:38:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:25965 - exited with rc=0 Feb 17 18:38:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:25965:stderr [ -- empty -- ] Feb 17 18:38:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:25965:stdout [ -- empty -- ] Feb 17 18:38:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:25965 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:38:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(25984) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:38:09 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.731ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.540ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.535ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:38:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:38:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:25984 - exited with rc=0 Feb 17 18:38:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:25984:stderr [ -- empty -- ] Feb 17 18:38:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:25984:stdout [ -- empty -- ] Feb 17 18:38:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:25984 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:38:10 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:38:10 INFO: MySQL is not running Feb 17 18:38:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:38:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:38:17 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.679ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.709ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.538ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.531ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:38:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:38:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:26498 - exited with rc=0 Feb 17 18:38:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:26498:stderr [ -- empty -- ] Feb 17 18:38:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:26498:stdout [ -- empty -- ] Feb 17 18:38:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:26498 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:38:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(26517) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:38:18 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.717ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.526ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.527ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:38:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:38:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:26517 - exited with rc=0 Feb 17 18:38:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:26517:stderr [ -- empty -- ] Feb 17 18:38:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:26517:stdout [ -- empty -- ] Feb 17 18:38:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:26517 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:38:18 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:38:18 INFO: MySQL is not running Feb 17 18:38:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:38:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:38:23 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.000333 (1 ticks in 30s) Feb 17 18:38:23 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.460000 (full: 0.46 0.61 0.31 3/458 27294) Feb 17 18:38:23 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 ocf-ns_IPaddr2: 2016/02/17_18:38:25 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.608ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.640ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.527ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.542ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:38:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:38:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:27202 - exited with rc=0 Feb 17 18:38:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:27202:stderr [ -- empty -- ] Feb 17 18:38:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:27202:stdout [ -- empty -- ] Feb 17 18:38:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:27202 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:38:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(27247) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:38:26 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.697ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.530ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.532ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:38:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:38:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:27247 - exited with rc=0 Feb 17 18:38:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:27247:stderr [ -- empty -- ] Feb 17 18:38:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:27247:stdout [ -- empty -- ] Feb 17 18:38:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:27247 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:38:26 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:38:26 INFO: MySQL is not running Feb 17 18:38:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:38:27 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:38:28 INFO: net.ipv4.ip_forward = 1 Feb 17 18:38:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:38:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:27585 - exited with rc=0 Feb 17 18:38:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:27585:stderr [ -- empty -- ] Feb 17 18:38:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:27585:stdout [ -- empty -- ] Feb 17 18:38:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:27585 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:38:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:38:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:38:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:38:33 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:38:33 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:38:33 INFO: haproxy daemon running Feb 17 18:38:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:38:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(27739) = 0: Resource temporarily unavailable (11) Feb 17 18:38:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(27758) = 0: Resource temporarily unavailable (11) Feb 17 18:38:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:27918 - exited with rc=0 Feb 17 18:38:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:27918:stderr [ -- empty -- ] Feb 17 18:38:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:27918:stdout [ -- empty -- ] Feb 17 18:38:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:27918 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:38:34 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.628ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.647ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.535ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.535ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:38:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:38:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:27739 - exited with rc=0 Feb 17 18:38:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:27739:stderr [ -- empty -- ] Feb 17 18:38:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:27739:stdout [ -- empty -- ] Feb 17 18:38:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:27739 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:38:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(27758) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:38:34 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.784ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.531ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.535ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:38:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:38:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:27758 - exited with rc=0 Feb 17 18:38:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:27758:stderr [ -- empty -- ] Feb 17 18:38:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:27758:stdout [ -- empty -- ] Feb 17 18:38:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:27758 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:38:34 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:38:34 INFO: MySQL is not running Feb 17 18:38:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:38:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:38:42 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.782ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.812ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.769ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.535ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:38:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:38:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:28297 - exited with rc=0 Feb 17 18:38:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:28297:stderr [ -- empty -- ] Feb 17 18:38:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:28297:stdout [ -- empty -- ] Feb 17 18:38:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:28297 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:38:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(28316) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:38:42 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.763ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.527ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:38:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:38:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:28316 - exited with rc=0 Feb 17 18:38:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:28316:stderr [ -- empty -- ] Feb 17 18:38:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:28316:stdout [ -- empty -- ] Feb 17 18:38:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:28316 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:38:42 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:38:42 INFO: MySQL is not running Feb 17 18:38:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:38:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:38:50 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.641ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.780ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.541ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.527ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:38:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:38:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:28784 - exited with rc=0 Feb 17 18:38:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:28784:stderr [ -- empty -- ] Feb 17 18:38:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:28784:stdout [ -- empty -- ] Feb 17 18:38:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:28784 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:38:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(28832) = 0: Resource temporarily unavailable (11) ocf-mysql-wss: 2016/02/17_18:38:50 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:38:50 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:38:50 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.717ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.527ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.528ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:38:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:38:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:28832 - exited with rc=0 Feb 17 18:38:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:28832:stderr [ -- empty -- ] Feb 17 18:38:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:28832:stdout [ -- empty -- ] Feb 17 18:38:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:28832 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:38:53 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.000000 (0 ticks in 30s) Feb 17 18:38:53 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.360000 (full: 0.36 0.57 0.31 2/457 29174) Feb 17 18:38:53 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:38:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:38:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:38:58 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:38:58 INFO: net.ipv4.ip_forward = 1 Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(29276) = 0: Resource temporarily unavailable (11) Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(29295) = 0: Resource temporarily unavailable (11) Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:29487 - exited with rc=0 Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:29487:stderr [ -- empty -- ] Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:29487:stdout [ -- empty -- ] Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:29487 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:38:58 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.758ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.786ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.527ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.528ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:29276 - exited with rc=0 Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:29276:stderr [ -- empty -- ] Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:29276:stdout [ -- empty -- ] Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:29276 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(29295) = 0: Resource temporarily unavailable (11) ocf-mysql-wss: 2016/02/17_18:38:58 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:38:58 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:38:58 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.945ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.527ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:29295 - exited with rc=0 Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:29295:stderr [ -- empty -- ] Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:29295:stdout [ -- empty -- ] Feb 17 18:38:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:29295 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:39:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:39:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:39:03 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:39:03 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:39:03 INFO: haproxy daemon running Feb 17 18:39:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:39:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(29843) = 0: Resource temporarily unavailable (11) Feb 17 18:39:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(29862) = 0: Resource temporarily unavailable (11) Feb 17 18:39:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:29885 - exited with rc=0 Feb 17 18:39:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:29885:stderr [ -- empty -- ] Feb 17 18:39:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:29885:stdout [ -- empty -- ] Feb 17 18:39:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:29885 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:39:06 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:39:06 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:39:06 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.607ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.633ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.531ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.526ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:39:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:39:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:29843 - exited with rc=0 Feb 17 18:39:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:29843:stderr [ -- empty -- ] Feb 17 18:39:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:29843:stdout [ -- empty -- ] Feb 17 18:39:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:29843 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(29862) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:39:06 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.804ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.534ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:39:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:39:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:29862 - exited with rc=0 Feb 17 18:39:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:29862:stderr [ -- empty -- ] Feb 17 18:39:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:29862:stdout [ -- empty -- ] Feb 17 18:39:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:29862 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:39:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:39:14 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:39:14 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:39:14 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.706ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.742ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.535ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.538ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:39:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:39:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:30369 - exited with rc=0 Feb 17 18:39:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:30369:stderr [ -- empty -- ] Feb 17 18:39:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:30369:stdout [ -- empty -- ] Feb 17 18:39:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:30369 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(30414) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:39:14 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.674ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.527ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.551ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:39:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:39:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:30414 - exited with rc=0 Feb 17 18:39:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:30414:stderr [ -- empty -- ] Feb 17 18:39:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:30414:stdout [ -- empty -- ] Feb 17 18:39:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:30414 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:39:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:39:22 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:39:22 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:39:22 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.722ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.743ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.556ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.530ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:39:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:39:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:31098 - exited with rc=0 Feb 17 18:39:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:31098:stderr [ -- empty -- ] Feb 17 18:39:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:31098:stdout [ -- empty -- ] Feb 17 18:39:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:31098 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(31117) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:39:22 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.663ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.533ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.539ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:39:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:39:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:31117 - exited with rc=0 Feb 17 18:39:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:31117:stderr [ -- empty -- ] Feb 17 18:39:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:31117:stdout [ -- empty -- ] Feb 17 18:39:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:31117 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:23 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.000000 (0 ticks in 30s) Feb 17 18:39:23 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.330000 (full: 0.33 0.54 0.31 3/453 31359) Feb 17 18:39:23 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:39:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:39:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:39:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:39:28 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:39:28 INFO: net.ipv4.ip_forward = 1 Feb 17 18:39:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:39:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(31635) = 0: Resource temporarily unavailable (11) Feb 17 18:39:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(31654) = 0: Resource temporarily unavailable (11) Feb 17 18:39:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:31673 - exited with rc=0 Feb 17 18:39:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:31673:stderr [ -- empty -- ] Feb 17 18:39:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:31673:stdout [ -- empty -- ] Feb 17 18:39:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:31673 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:39:30 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:39:30 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:39:30 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.730ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.755ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.524ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.527ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:39:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:39:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:31635 - exited with rc=0 Feb 17 18:39:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:31635:stderr [ -- empty -- ] Feb 17 18:39:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:31635:stdout [ -- empty -- ] Feb 17 18:39:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:31635 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(31654) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:39:30 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.659ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.551ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:39:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:39:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:31654 - exited with rc=0 Feb 17 18:39:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:31654:stderr [ -- empty -- ] Feb 17 18:39:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:31654:stdout [ -- empty -- ] Feb 17 18:39:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:31654 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:39:33 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:39:33 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:39:33 INFO: haproxy daemon running Feb 17 18:39:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:39:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:32053 - exited with rc=0 Feb 17 18:39:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:32053:stderr [ -- empty -- ] Feb 17 18:39:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:32053:stdout [ -- empty -- ] Feb 17 18:39:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:32053 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:39:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:39:38 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:39:38 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:39:38 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.630ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.665ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.556ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.544ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:39:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:39:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:32212 - exited with rc=0 Feb 17 18:39:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:32212:stderr [ -- empty -- ] Feb 17 18:39:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:32212:stdout [ -- empty -- ] Feb 17 18:39:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:32212 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(32231) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:39:38 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.783ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.547ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.543ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:39:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:39:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:32231 - exited with rc=0 Feb 17 18:39:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:32231:stderr [ -- empty -- ] Feb 17 18:39:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:32231:stdout [ -- empty -- ] Feb 17 18:39:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:32231 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:43 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:39:43 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:39:46 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:39:46 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:39:46 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.746ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.977ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.528ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.528ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:39:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Interrupted system call (4) Feb 17 18:39:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:32674 - exited with rc=0 Feb 17 18:39:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:32674:stderr [ -- empty -- ] Feb 17 18:39:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:32674:stdout [ -- empty -- ] Feb 17 18:39:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:32674 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(32693) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:39:46 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.679ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.529ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:39:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11979) = 0: Resource temporarily unavailable (11) Feb 17 18:39:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:32693 - exited with rc=0 Feb 17 18:39:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:32693:stderr [ -- empty -- ] Feb 17 18:39:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:32693:stdout [ -- empty -- ] Feb 17 18:39:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:32693 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: warning: child_timeout_callback: p_mysql_start_0 process (PID 11979) timed out Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: warning: operation_finished: p_mysql_start_0:11979 - timed out after 300000ms Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:11979:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:11979:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:11979:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:11979:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_start_0:11979:stdout [ -- empty -- ] Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_mysql action:start call_id:68 pid:11979 exit-code:1 exec-time:300003ms queue-time:0ms Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_mysql after start op Timed Out (interval=0) Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed mysql-wss_meta-data_0 process 554 exited with rc=0 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/32) Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: error: process_lrm_event: Operation p_mysql_start_0: Timed Out (node=node-3.test.domain.local, call=68, timeout=300000ms) Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: node-3.test.domain.local-p_mysql_start_0:68 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n ] Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_mysql' with start op Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.483 2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.484 (null) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=484 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_mysql']/lrm_rsc_op[@id='p_mysql_last_0']: @operation_key=p_mysql_start_0, @operation=start, @transition-key=77:79:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=2:1;77:79:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=68, @rc-code=1, @op-status=2, @last-run=1455734088, @last-rc-change=1455734088, @exec-time=300003 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_mysql']: Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/45, version=0.155.485) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='last-failure-p_mysql'] does not exist Feb 17 18:39:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.485 to 0.155.484 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 47: last-failure-p_mysql=1455734388 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dcd0 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 45 for fail-count-p_mysql=1 passed Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/47) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349db20 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: fail-count-p_mysql=value++ Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: value++, Current: 1, Stored: 1 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: info: attrd_local_callback: Expanded fail-count-p_mysql=value++ to 2 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of fail-count-p_mysql is 2 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-p_mysql (2) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349db20 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.485 2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.486 (null) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=486 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/47, version=0.155.486) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='fail-count-p_mysql'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[8]) Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 49: fail-count-p_mysql=2 Feb 17 18:39:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.486 to 0.155.485 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 47 for last-failure-p_mysql=1455734388 passed Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/49) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dbc0 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.486 2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.487 (null) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=487 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-fail-count-p_mysql']: @value=2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/49, version=0.155.487) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dbc0 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.487 to 0.155.486 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: last-failure-p_mysql=1455734388 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1455734388, Current: 1455734388, Stored: 1455734388 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 49 for fail-count-p_mysql=2 passed Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dbc0 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/741, version=0.155.487) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dbc0 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: fail-count-p_mysql=value++ Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: value++, Current: 2, Stored: 2 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: info: attrd_local_callback: Expanded fail-count-p_mysql=value++ to 3 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of fail-count-p_mysql is 3 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-p_mysql (3) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='fail-count-p_mysql'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[8]) Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 51: fail-count-p_mysql=3 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/51) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d940 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.487 2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.488 (null) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=488 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-fail-count-p_mysql']: @value=3 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/51, version=0.155.488) Feb 17 18:39:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.488 to 0.155.487 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d940 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 51 for fail-count-p_mysql=3 passed Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d940 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/743, version=0.155.488) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d940 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/745, version=0.155.488) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/747, version=0.155.488) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: last-failure-p_mysql=1455734388 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1455734388, Current: 1455734388, Stored: 1455734388 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: fail-count-p_mysql=value++ Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: value++, Current: 3, Stored: 3 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: info: attrd_local_callback: Expanded fail-count-p_mysql=value++ to 4 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of fail-count-p_mysql is 4 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-p_mysql (4) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='fail-count-p_mysql'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[8]) Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 53: fail-count-p_mysql=4 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/53) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d940 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: last-failure-p_mysql=1455734388 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1455734388, Current: 1455734388, Stored: 1455734388 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.488 2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.489 (null) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=489 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-fail-count-p_mysql']: @value=4 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/53, version=0.155.489) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.489 to 0.155.488 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d940 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 53 for fail-count-p_mysql=4 passed Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/749, version=0.155.489) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_neutron-plugin-openvswitch-agent' not found (12 active resources) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_neutron-plugin-openvswitch-agent:1' not found (12 active resources) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_neutron-plugin-openvswitch-agent' to the rsc list (13 active resources) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=13:85:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-plugin-openvswitch-agent_monitor_0 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=75, reply=1, notify=0, exit=4202464 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:75 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_neutron-l3-agent' not found (13 active resources) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_neutron-l3-agent:1' not found (13 active resources) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_neutron-l3-agent' to the rsc list (14 active resources) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=14:85:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-l3-agent_monitor_0 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=80, reply=1, notify=0, exit=4202464 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_neutron-l3-agent action:monitor call_id:80 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_neutron-dhcp-agent' not found (14 active resources) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_neutron-dhcp-agent:1' not found (14 active resources) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_neutron-dhcp-agent' to the rsc list (15 active resources) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=15:85:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-dhcp-agent_monitor_0 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=85, reply=1, notify=0, exit=4202464 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_neutron-dhcp-agent action:monitor call_id:85 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_neutron-metadata-agent' not found (15 active resources) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_neutron-metadata-agent:1' not found (15 active resources) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_neutron-metadata-agent' to the rsc list (16 active resources) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=16:85:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-metadata-agent_monitor_0 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=90, reply=1, notify=0, exit=4202464 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_neutron-metadata-agent action:monitor call_id:90 ocf-neutron-ovs-agent: 2016/02/17_18:39:48 DEBUG: Create pid file dir: /var/run/resource-agents/ocf-neutron-ovs-agent and chown to neutron ocf-neutron-l3-agent: 2016/02/17_18:39:48 DEBUG: Create pid file dir: /var/run/resource-agents/ocf-neutron-l3-agent and chown to neutron ocf-neutron-ovs-agent: 2016/02/17_18:39:48 WARNING: OpenStack Neutron agent 'neutron-openvswitch-agent' not running. Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_0:590 - exited with rc=7 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_0:590:stderr [ -- empty -- ] Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_0:590:stdout [ -- empty -- ] Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:75 pid:590 exit-code:7 exec-time:149ms queue-time:0ms Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-plugin-openvswitch-agent after monitor op complete (interval=0) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(596) = 0: Resource temporarily unavailable (11) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(609) = 0: Resource temporarily unavailable (11) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(615) = 0: Resource temporarily unavailable (11) ocf-neutron-dhcp-agent: 2016/02/17_18:39:48 DEBUG: Create pid file dir: /var/run/resource-agents/ocf-neutron-dhcp-agent and chown to neutron Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ocf-neutron-ovs-agent_meta-data_0 process 718 exited with rc=0 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/33) Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-plugin-openvswitch-agent_monitor_0: not running (node=node-3.test.domain.local, call=75, rc=7, cib-update=33, confirmed=true) Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-plugin-openvswitch-agent' with monitor op ocf-neutron-l3-agent: 2016/02/17_18:39:48 WARNING: OpenStack Neutron agent 'neutron-l3-agent' not running. Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_0:596 - exited with rc=7 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_0:596:stderr [ -- empty -- ] Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_0:596:stdout [ -- empty -- ] Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:80 pid:596 exit-code:7 exec-time:149ms queue-time:1ms Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-l3-agent after monitor op complete (interval=0) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(609) = 0: Resource temporarily unavailable (11) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(615) = 0: Resource temporarily unavailable (11) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-neutron-metadata-agent: 2016/02/17_18:39:48 DEBUG: Create pid file dir: /var/run/resource-agents/ocf-neutron-metadata-agent and chown to neutron Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.489 2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.490 (null) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=490 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/33, version=0.155.490) Feb 17 18:39:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.490 to 0.155.489 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ocf-neutron-l3-agent_meta-data_0 process 730 exited with rc=0 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: get_rsc_restart_list: Attr dummy is not reloadable Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-l3-agent_monitor_0: not running (node=node-3.test.domain.local, call=80, rc=7, cib-update=34, confirmed=true) Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-l3-agent' with monitor op Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.490 2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.491 (null) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=491 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_dns']/lrm_rsc_op[@id='p_dns_last_0']: @operation_key=p_dns_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=115:85:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;115:85:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=257, @last-run=1455734388, @last-rc-change=1455734388, @exec-time=167 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/351, version=0.155.491) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/34) Feb 17 18:39:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.491 to 0.155.490 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.491 2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.492 (null) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=492 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/34, version=0.155.492) Feb 17 18:39:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.492 to 0.155.491 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.492 2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.493 (null) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=493 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_mysql']/lrm_rsc_op[@id='p_mysql_last_failure_0']: @operation_key=p_mysql_monitor_60000, @operation=monitor, @crm-debug-origin=do_update_resource, @transition-key=76:85:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:1;76:85:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=254, @op-status=0, @interval=60000, @last-rc-change=1455 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/352, version=0.155.493) Feb 17 18:39:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.493 to 0.155.492 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-neutron-dhcp-agent: 2016/02/17_18:39:48 WARNING: OpenStack Neutron agent 'neutron-dhcp-agent' not running. Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_0:609 - exited with rc=7 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_0:609:stderr [ -- empty -- ] Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_0:609:stdout [ -- empty -- ] Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:85 pid:609 exit-code:7 exec-time:178ms queue-time:0ms Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-dhcp-agent after monitor op complete (interval=0) Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(615) = 0: Resource temporarily unavailable (11) ocf-neutron-metadata-agent: 2016/02/17_18:39:48 WARNING: OpenStack Neutron agent 'neutron-metadata-agent' not running. Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_0:615 - exited with rc=7 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_0:615:stderr [ -- empty -- ] Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_0:615:stdout [ -- empty -- ] Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-metadata-agent action:monitor call_id:90 pid:615 exit-code:7 exec-time:166ms queue-time:0ms Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d940 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ocf-neutron-dhcp-agent_meta-data_0 process 760 exited with rc=0 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: get_rsc_restart_list: Attr dummy is not reloadable Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-dhcp-agent_monitor_0: not running (node=node-3.test.domain.local, call=85, rc=7, cib-update=35, confirmed=true) Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-dhcp-agent' with monitor op Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d940 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.493 2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.494 (null) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=494 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-fail-count-p_mysql']: @value=37 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/751, version=0.155.494) Feb 17 18:39:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.494 to 0.155.493 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-metadata-agent after monitor op complete (interval=0) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.494 2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.495 (null) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=495 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-last-failure-p_mysql']: @value=1455734388 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/753, version=0.155.495) Feb 17 18:39:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.495 to 0.155.494 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.495 2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.496 (null) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=496 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-fail-count-p_mysql']: @value=38 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/755, version=0.155.496) Feb 17 18:39:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.496 to 0.155.495 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ocf-neutron-metadata-agent_meta-data_0 process 768 exited with rc=0 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-metadata-agent_monitor_0: not running (node=node-3.test.domain.local, call=90, rc=7, cib-update=36, confirmed=true) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-metadata-agent' with monitor op Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/757, version=0.155.496) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/35) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/36) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='fail-count-p_mysql'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[8]) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/55) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d940 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-28) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a20f0 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d940 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.496 2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.497 (null) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=497 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/35, version=0.155.497) Feb 17 18:39:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.497 to 0.155.496 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-28-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-28-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-28-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-28) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-28) state:2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a20f0 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-28-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-28-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-28-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a6660 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.497 2 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.498 (null) Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=498 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/36, version=0.155.498) Feb 17 18:39:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.498 to 0.155.497 Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:39:48 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=213:85:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_notify_0 Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=91, reply=1, notify=0, exit=4202464 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:48 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_conntrackd action:notify call_id:91 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-27) state:2 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a6660 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-27) Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a6730 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/55, version=0.155.498) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.498 2 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.499 (null) Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=499 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_heat-engine']/lrm_rsc_op[@id='p_heat-engine_last_0']: @operation_key=p_heat-engine_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=123:85:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;123:85:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=258, @last-run=1455734388, @last-rc-change=145573438 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/353, version=0.155.499) Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:49 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.499 to 0.155.498 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='last-failure-p_mysql'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[9]) Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 55 for fail-count-p_mysql=4 passed Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/57) Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:772 - exited with rc=0 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:772:stderr [ -- empty -- ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:772:stdout [ -- empty -- ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_conntrackd action:notify call_id:91 pid:772 exit-code:0 exec-time:25ms queue-time:0ms Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-27) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-27) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a6730 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a3080 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=225:85:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_rabbitmq-server_notify_0 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-28) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=92, reply=1, notify=0, exit=4202464 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_rabbitmq-server action:notify call_id:92 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d940 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a3080 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-28-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-28-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-28-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/57, version=0.155.499) Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='fail-count-p_mysql'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[8]) Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 57 for last-failure-p_mysql=1455734388 passed Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/59) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-28) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-28) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d940 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-28-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-28-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-28-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a20f0 Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_conntrackd after notify op complete (interval=0) Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_conntrackd_notify_0 from 213:85:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455734389-13 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='last-failure-p_mysql'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[9]) Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_notify_0: ok (node=node-3.test.domain.local, call=91, rc=0, cib-update=0, confirmed=true) Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/61) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a20f0 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-27) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a20f0 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.499 2 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.500 (null) Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=500 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_dns']: word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/59, version=0.155.500) Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 59 for fail-count-p_mysql=4 passed Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-27) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-27) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a20f0 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a2ca0 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/61, version=0.155.500) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-28) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d940 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: rabbit-master= Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null) Feb 17 18:39:49 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 61 for last-failure-p_mysql=1455734388 passed Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a2ca0 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-28-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-28-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-28-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-28) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-28) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d940 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-28-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-28-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-28-header Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-797-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [797] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761349cf10, cpd=0x7f761349d7f4 Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f761349cf10 Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-6268-797-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-797-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-797-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-797-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-797-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-797-26-header Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-6268-797-26-header Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-6268-797-26-header Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-797-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [797] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-797-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-797-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-797-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-797-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-797-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-797-26-header Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-797-26-header Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-797-26-header Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:39:49 [797] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 lrmd: 2016/02/17_18:39:49 INFO: p_rabbitmq-server: notify: pre-promote begin. lrmd: 2016/02/17_18:39:49 INFO: p_rabbitmq-server: my_host(): hostlist is: node-2.test.domain.local lrmd: 2016/02/17_18:39:49 DEBUG: p_rabbitmq-server: my_host(): comparing 'messaging-node-3' with 'messaging-node-2' Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_notify_0:781 - exited with rc=0 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:781:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:781:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:781:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:781:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:781:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:781:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_notify_0:781:stdout [ -- empty -- ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_rabbitmq-server action:notify call_id:92 pid:781 exit-code:0 exec-time:156ms queue-time:1ms Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_rabbitmq-server after notify op complete (interval=0) Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_rabbitmq-server_notify_0 from 225:85:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455734389-14 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_rabbitmq-server_notify_0: ok (node=node-3.test.domain.local, call=92, rc=0, cib-update=0, confirmed=true) Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_rabbitmq-server_notify_0:92 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n ] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'sysinfo_node-3.test.domain.local' not found (16 active resources) Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'sysinfo_node-3.test.domain.local' to the rsc list (17 active resources) Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=16:86:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=sysinfo_node-3.test.domain.local_monitor_0 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=96, reply=1, notify=0, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:96 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_0:883 - exited with rc=7 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: sysinfo_node-3.test.domain.local_monitor_0:883:stderr [ rm: cannot remove '/var/run//SysInfo-sysinfo_node-3.test.domain.local': No such file or directory ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_0:883:stdout [ -- empty -- ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:96 pid:883 exit-code:7 exec-time:7ms queue-time:0ms Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'sysinfo_node-2.test.domain.local' not found (17 active resources) Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'sysinfo_node-2.test.domain.local' to the rsc list (18 active resources) Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=17:86:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=sysinfo_node-2.test.domain.local_monitor_0 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=100, reply=1, notify=0, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:sysinfo_node-2.test.domain.local action:monitor call_id:100 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource sysinfo_node-3.test.domain.local after monitor op complete (interval=0) Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation sysinfo_node-3.test.domain.local_monitor_0: not running (node=node-3.test.domain.local, call=96, rc=7, cib-update=37, confirmed=true) Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: node-3.test.domain.local-sysinfo_node-3.test.domain.local_monitor_0:96 [ rm: cannot remove '/var/run//SysInfo-sysinfo_node-3.test.domain.local': No such file or directory\n ] Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'sysinfo_node-3.test.domain.local' with monitor op Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/37) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-2.test.domain.local_monitor_0:890 - exited with rc=7 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: sysinfo_node-2.test.domain.local_monitor_0:890:stderr [ rm: cannot remove '/var/run//SysInfo-sysinfo_node-2.test.domain.local': No such file or directory ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-2.test.domain.local_monitor_0:890:stdout [ -- empty -- ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-2.test.domain.local action:monitor call_id:100 pid:890 exit-code:7 exec-time:7ms queue-time:0ms Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d940 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.500 2 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.501 (null) Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=501 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/37, version=0.155.501) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:49 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.501 to 0.155.500 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d940 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'ping_vip__public' not found (18 active resources) Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'ping_vip__public:1' not found (18 active resources) Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'ping_vip__public' to the rsc list (19 active resources) Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=18:86:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=ping_vip__public_monitor_0 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=105, reply=1, notify=0, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:ping_vip__public action:monitor call_id:105 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource sysinfo_node-2.test.domain.local after monitor op complete (interval=0) Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation sysinfo_node-2.test.domain.local_monitor_0: not running (node=node-3.test.domain.local, call=100, rc=7, cib-update=38, confirmed=true) Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: node-3.test.domain.local-sysinfo_node-2.test.domain.local_monitor_0:100 [ rm: cannot remove '/var/run//SysInfo-sysinfo_node-2.test.domain.local': No such file or directory\n ] Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'sysinfo_node-2.test.domain.local' with monitor op Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/38) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_0:897 - exited with rc=7 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_0:897:stderr [ -- empty -- ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_0:897:stdout [ -- empty -- ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:ping_vip__public action:monitor call_id:105 pid:897 exit-code:7 exec-time:6ms queue-time:0ms Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d940 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_ntp' not found (19 active resources) Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_ntp:1' not found (19 active resources) Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_ntp' to the rsc list (20 active resources) Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=19:86:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_ntp_monitor_0 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=110, reply=1, notify=0, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_ntp action:monitor call_id:110 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource ping_vip__public after monitor op complete (interval=0) Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.501 2 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.502 (null) Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=502 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/38, version=0.155.502) Feb 17 18:39:49 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.502 to 0.155.501 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ping_meta-data_0 process 903 exited with rc=0 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/39) Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation ping_vip__public_monitor_0: not running (node=node-3.test.domain.local, call=105, rc=7, cib-update=39, confirmed=true) Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'ping_vip__public' with monitor op Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d940 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d940 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=211:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_notify_0 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=111, reply=1, notify=0, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_conntrackd action:notify call_id:111 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.502 2 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.503 (null) Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=503 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/39, version=0.155.503) Feb 17 18:39:49 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.503 to 0.155.502 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d940 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-ns_ntp: 2016/02/17_18:39:49 INFO: ntp daemon is not running Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_0:902 - exited with rc=7 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_0:902:stderr [ -- empty -- ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_0:902:stdout [ -- empty -- ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:110 pid:902 exit-code:7 exec-time:29ms queue-time:1ms Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(913) = 0: Resource temporarily unavailable (11) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=223:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_rabbitmq-server_notify_0 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=112, reply=1, notify=0, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_rabbitmq-server action:notify call_id:112 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_ntp after monitor op complete (interval=0) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_ntp_meta-data_0 process 919 exited with rc=0 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/40) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.503 2 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.504 (null) Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=504 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/40, version=0.155.504) Feb 17 18:39:49 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.504 to 0.155.503 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_ntp_monitor_0: not running (node=node-3.test.domain.local, call=110, rc=7, cib-update=40, confirmed=true) Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_ntp' with monitor op Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_dns_start_0 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=114:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_dns_start_0 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=113, reply=1, notify=0, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_dns action:start call_id:113 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:913 - exited with rc=0 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:913:stderr [ -- empty -- ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:913:stdout [ -- empty -- ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_conntrackd action:notify call_id:111 pid:913 exit-code:0 exec-time:54ms queue-time:0ms Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(918) = 0: Resource temporarily unavailable (11) Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(933) = 0: Resource temporarily unavailable (11) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a6760 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a6760 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a6660 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d940 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_heat-engine_start_0 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=123:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_heat-engine_start_0 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=114, reply=1, notify=0, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_heat-engine action:start call_id:114 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a6660 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.504 2 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.505 (null) Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=505 Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_heat-engine']: word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1115) = 0: Resource temporarily unavailable (11) Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1186) = 0: Resource temporarily unavailable (11) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_rabbitmq-server after notify op complete (interval=0) Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_rabbitmq-server_notify_0 from 223:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455734389-16 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_rabbitmq-server_notify_0: ok (node=node-3.test.domain.local, call=112, rc=0, cib-update=0, confirmed=true) Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_rabbitmq-server_notify_0:112 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n ] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-heat-engine: 2016/02/17_18:39:49 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1115) = 0: Resource temporarily unavailable (11) Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:1186 - exited with rc=0 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:1186:stderr [ -- empty -- ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:1186:stdout [ -- empty -- ] Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:1186 exit-code:0 exec-time:143ms queue-time:1ms Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=132:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=119, reply=1, notify=0, exit=4202464 Feb 17 18:39:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_heat-engine after monitor op complete (interval=20000) Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_heat-engine_monitor_20000: ok (node=node-3.test.domain.local, call=118, rc=0, cib-update=45, confirmed=false) Feb 17 18:39:49 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_heat-engine' with monitor op Feb 17 18:39:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/45) Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349db20 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.510 2 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.511 (null) Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=511 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_heat-engine']: Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=1309 id=c43d0a63-a7ad-45e6-b7df-8e86677f6996 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-1309-11) Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [1309] Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: master-p_conntrackd=1000 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1000, Current: 100, Stored: 100 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of master-p_conntrackd is 1000 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_conntrackd (1000) Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_conntrackd'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[2]) Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/63) Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 63: master-p_conntrackd=1000 Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: master-p_conntrackd=1000 for node-3.test.domain.local Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: info: main: Update master-p_conntrackd=1000 sent via attrd Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-1309-14) Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-1309-14) state:2 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-1309-14-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-1309-14-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-1309-14-header Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-1309-14-header Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-1309-14-header Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-1309-14-header Feb 17 18:39:50 [1309] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349de80 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-1309-11) Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-1309-11) state:2 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-1309-11-header Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-1309-11-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-1309-11-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349de80 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.512 2 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.513 (null) Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=513 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-master-p_conntrackd']: @value=1000 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/63, version=0.155.513) Feb 17 18:39:50 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.513 to 0.155.512 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 63 for master-p_conntrackd=1000 passed Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_promote_0:1115 - exited with rc=0 Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_promote_0:1115:stderr [ -- empty -- ] Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_promote_0:1115:stdout [ -- empty -- ] Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_conntrackd action:promote call_id:116 pid:1115 exit-code:0 exec-time:625ms queue-time:9ms Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_conntrackd after promote op complete (interval=0) Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_promote_0: ok (node=node-3.test.domain.local, call=116, rc=0, cib-update=46, confirmed=true) Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_conntrackd' with promote op Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/46) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1247) = 0: Resource temporarily unavailable (11) Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1270) = 0: Resource temporarily unavailable (11) Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1287) = 0: Resource temporarily unavailable (11) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.513 2 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.514 (null) Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=514 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_conntrackd']/lrm_rsc_op[@id='p_conntrackd_last_0']: @operation_key=p_conntrackd_promote_0, @operation=promote, @transition-key=39:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;39:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=116, @last-run=1455734389, @last-rc-change=1455734389, @exec-time=625, @queue-time=9 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/46, version=0.155.514) Feb 17 18:39:50 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.514 to 0.155.513 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:1247 - exited with rc=0 Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:1247:stderr [ -- empty -- ] Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:1247:stdout [ -- empty -- ] Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:1247 exit-code:0 exec-time:312ms queue-time:0ms Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-plugin-openvswitch-agent after monitor op complete (interval=20000) Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-plugin-openvswitch-agent_monitor_20000: ok (node=node-3.test.domain.local, call=119, rc=0, cib-update=47, confirmed=false) Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-plugin-openvswitch-agent' with monitor op Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1270) = 0: Resource temporarily unavailable (11) Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1287) = 0: Resource temporarily unavailable (11) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/759, version=0.155.514) Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/47) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.514 2 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.515 (null) Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=515 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_neutron-plugin-openvswitch-agent']: Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/65, version=0.155.516) Feb 17 18:39:50 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.516 to 0.155.515 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 65 for probe_complete=true passed Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df50 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-neutron-dhcp-agent: 2016/02/17_18:39:50 WARNING: OpenStack Neutron agent 'neutron-dhcp-agent' not running. Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=212:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_notify_0 Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=122, reply=1, notify=0, exit=4202464 Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_conntrackd action:notify call_id:122 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df00 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/761, version=0.155.516) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df00 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1270) = 0: Resource temporarily unavailable (11) Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1287) = 0: Resource temporarily unavailable (11) Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:1381 - exited with rc=0 Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:1381:stderr [ -- empty -- ] Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:1381:stdout [ -- empty -- ] Feb 17 18:39:50 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_conntrackd action:notify call_id:122 pid:1381 exit-code:0 exec-time:125ms queue-time:1ms Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_conntrackd after notify op complete (interval=0) Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_conntrackd_notify_0 from 212:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455734390-17 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_notify_0: ok (node=node-3.test.domain.local, call=122, rc=0, cib-update=0, confirmed=true) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:50 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: rabbit-master= Feb 17 18:39:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.516 2 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.517 (null) Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=517 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_neutron-metadata-agent']/lrm_rsc_op[@id='p_neutron-metadata-agent_last_0']: @operation_key=p_neutron-metadata-agent_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=153:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;153:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=267, @last-run=1455 Feb 17 18:39:50 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/359, version=0.155.517) Feb 17 18:39:50 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.517 to 0.155.516 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:39:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.517 2 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.518 (null) Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=518 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_neutron-dhcp-agent']/lrm_rsc_op[@id='p_neutron-dhcp-agent_last_0']: @operation_key=p_neutron-dhcp-agent_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=145:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;145:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=265, @last-run=1455734389, @las Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/360, version=0.155.518) Feb 17 18:39:52 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.518 to 0.155.517 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.518 2 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.519 (null) Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=519 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_neutron-l3-agent']/lrm_rsc_op[@id='p_neutron-l3-agent_last_0']: @operation_key=p_neutron-l3-agent_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=137:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;137:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=264, @last-run=1455734389, @last-rc-c Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/361, version=0.155.519) Feb 17 18:39:52 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.519 to 0.155.518 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-neutron-l3-agent: 2016/02/17_18:39:52 DEBUG: Create pid file: /var/run/resource-agents/ocf-neutron-l3-agent/ocf-neutron-l3-agent.pid with content ocf-neutron-l3-agent: 2016/02/17_18:39:52 WARNING: OpenStack Neutron agent (neutron-l3-agent) was run, but no PID file found. ocf-neutron-l3-agent: 2016/02/17_18:39:52 WARNING: Writing PID='1551' to '/var/run/resource-agents/ocf-neutron-l3-agent/ocf-neutron-l3-agent.pid' for 'neutron-l3-agent' worker... ocf-neutron-l3-agent: 2016/02/17_18:39:52 INFO: OpenStack L3 agent (neutron-l3-agent) started Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_start_0:1270 - exited with rc=0 Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_neutron-l3-agent_start_0:1270:stderr [ No handlers could be found for logger "oslo_config.cfg" ] Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_neutron-l3-agent_start_0:1270:stderr [ sh: 1: cannot create : Directory nonexistent ] Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_neutron-l3-agent_start_0:1270:stderr [ cat: /var/run/resource-agents/ocf-neutron-l3-agent/ocf-neutron-l3-agent.pid: No such file or directory ] Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_start_0:1270:stdout [ -- empty -- ] Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_neutron-l3-agent action:start call_id:120 pid:1270 exit-code:0 exec-time:2564ms queue-time:0ms Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1287) = 0: Resource temporarily unavailable (11) Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1424) = 0: Resource temporarily unavailable (11) Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1449) = 0: Resource temporarily unavailable (11) Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-l3-agent after start op complete (interval=0) Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-l3-agent_start_0: ok (node=node-3.test.domain.local, call=120, rc=0, cib-update=48, confirmed=true) Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_neutron-l3-agent_start_0:120 [ No handlers could be found for logger "oslo_config.cfg"\nsh: 1: cannot create : Directory nonexistent\ncat: /var/run/resource-agents/ocf-neutron-l3-agent/ocf-neutron-l3-agent.pid: No such file or directory\n ] Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-l3-agent' with start op Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/48) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.519 2 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.520 (null) Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=520 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_neutron-l3-agent']/lrm_rsc_op[@id='p_neutron-l3-agent_last_0']: @operation_key=p_neutron-l3-agent_start_0, @operation=start, @transition-key=139:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;139:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=120, @rc-code=0, @last-run=1455734390, @last-rc-change=1455734390, @exec-ti Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/48, version=0.155.520) Feb 17 18:39:52 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.520 to 0.155.519 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-neutron-dhcp-agent: 2016/02/17_18:39:52 DEBUG: Create pid file: /var/run/resource-agents/ocf-neutron-dhcp-agent/ocf-neutron-dhcp-agent.pid with content ocf-neutron-dhcp-agent: 2016/02/17_18:39:52 WARNING: OpenStack Neutron agent (neutron-dhcp-agent) was run, but no PID file found. ocf-neutron-dhcp-agent: 2016/02/17_18:39:52 WARNING: Writing PID='1595' to '/var/run/resource-agents/ocf-neutron-dhcp-agent/ocf-neutron-dhcp-agent.pid' for 'neutron-dhcp-agent' worker... ocf-neutron-dhcp-agent: 2016/02/17_18:39:52 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded ocf-neutron-dhcp-agent: 2016/02/17_18:39:52 INFO: OpenStack DHCP Agent (neutron-dhcp-agent) started Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_start_0:1287 - exited with rc=0 Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_neutron-dhcp-agent_start_0:1287:stderr [ No handlers could be found for logger "oslo_config.cfg" ] Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_neutron-dhcp-agent_start_0:1287:stderr [ sh: 1: cannot create : Directory nonexistent ] Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_neutron-dhcp-agent_start_0:1287:stderr [ cat: /var/run/resource-agents/ocf-neutron-dhcp-agent/ocf-neutron-dhcp-agent.pid: No such file or directory ] Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_start_0:1287:stdout [ -- empty -- ] Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_neutron-dhcp-agent action:start call_id:121 pid:1287 exit-code:0 exec-time:2737ms queue-time:0ms Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1424) = 0: Resource temporarily unavailable (11) Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1449) = 0: Resource temporarily unavailable (11) Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-dhcp-agent after start op complete (interval=0) Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-dhcp-agent_start_0: ok (node=node-3.test.domain.local, call=121, rc=0, cib-update=49, confirmed=true) Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_neutron-dhcp-agent_start_0:121 [ No handlers could be found for logger "oslo_config.cfg"\nsh: 1: cannot create : Directory nonexistent\ncat: /var/run/resource-agents/ocf-neutron-dhcp-agent/ocf-neutron-dhcp-agent.pid: No such file or directory\n ] Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-dhcp-agent' with start op Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/49) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:52 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.520 2 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.521 (null) Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=521 Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_neutron-dhcp-agent']/lrm_rsc_op[@id='p_neutron-dhcp-agent_last_0']: @operation_key=p_neutron-dhcp-agent_start_0, @operation=start, @transition-key=147:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;147:86:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=121, @rc-code=0, @last-run=1455734390, @last-rc-change=1455734390, @e Feb 17 18:39:52 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/49, version=0.155.521) Feb 17 18:39:52 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.521 to 0.155.520 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:52 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=42:87:8:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_monitor_27000 Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=123, reply=1, notify=0, exit=4202464 Feb 17 18:39:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_conntrackd action:monitor call_id:123 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:52 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=143:87:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-l3-agent_monitor_20000 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=124, reply=1, notify=0, exit=4202464 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_neutron-l3-agent action:monitor call_id:124 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=153:87:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-dhcp-agent_monitor_20000 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=125, reply=1, notify=0, exit=4202464 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_neutron-dhcp-agent action:monitor call_id:125 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for sysinfo_node-3.test.domain.local_start_0 Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=167:87:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=sysinfo_node-3.test.domain.local_start_0 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=126, reply=1, notify=0, exit=4202464 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:sysinfo_node-3.test.domain.local action:start call_id:126 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.002667 (8 ticks in 30s) Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.870000 (full: 0.87 0.64 0.35 14/482 1677) Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.521 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.522 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=522 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_neutron-l3-agent']: word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.522 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.523 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=523 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_neutron-dhcp-agent']: word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.523 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.524 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=524 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_neutron-metadata-agent']: word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1726] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1726] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1726] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: (null), Stored: (null) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of arch is x86_64 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: arch (x86_64) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='arch'] does not exist Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 67: arch=x86_64 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/67) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [1726] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:39:53 [1726] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e110 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-1726-11) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-1726-11) state:2 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-1726-11-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-1726-11-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-1726-11-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.525 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.526 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=526 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/67, version=0.155.526) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 67 for arch=x86_64 passed Feb 17 18:39:53 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.526 to 0.155.525 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e110 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1424) = 0: Resource temporarily unavailable (11) Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1449) = 0: Resource temporarily unavailable (11) Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1639) = 0: Resource temporarily unavailable (11) Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:1644 - exited with rc=0 Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:1644:stderr [ -- empty -- ] Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:1644:stdout [ -- empty -- ] Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:1644 exit-code:0 exec-time:384ms queue-time:0ms Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-l3-agent after monitor op complete (interval=20000) Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-l3-agent_monitor_20000: ok (node=node-3.test.domain.local, call=124, rc=0, cib-update=50, confirmed=false) Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-l3-agent' with monitor op Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/50) Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1654) = 0: Resource temporarily unavailable (11) Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1666) = 0: Resource temporarily unavailable (11) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.526 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.527 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=527 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_neutron-l3-agent']: word_size:263168 Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1424) = 0: Resource temporarily unavailable (11) Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1449) = 0: Resource temporarily unavailable (11) Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1639) = 0: Resource temporarily unavailable (11) Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:1654 - exited with rc=0 Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:1654:stderr [ -- empty -- ] Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:1654:stdout [ -- empty -- ] Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:1654 exit-code:0 exec-time:364ms queue-time:0ms Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-dhcp-agent after monitor op complete (interval=20000) Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-dhcp-agent_monitor_20000: ok (node=node-3.test.domain.local, call=125, rc=0, cib-update=51, confirmed=false) Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-dhcp-agent' with monitor op Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1666) = 0: Resource temporarily unavailable (11) Feb 17 18:39:53 [1762] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=1762 id=29dd03c7-f7a4-4d53-8a71-940928da4d8d Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-1762-11) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [1762] Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/763, version=0.155.527) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/51) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [1762] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1762] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1762] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: (null), Stored: (null) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of os is Linux-3.13.0-77-generic Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: os (Linux-3.13.0-77-generic) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='os'] does not exist Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 69: os=Linux-3.13.0-77-generic Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/69) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [1762] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:39:53 [1762] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-1762-11) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-1762-11) state:2 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-1762-11-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-1762-11-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-1762-11-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1424) = 0: Resource temporarily unavailable (11) Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1449) = 0: Resource temporarily unavailable (11) Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:1639 - exited with rc=8 Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:1639:stderr [ -- empty -- ] Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:1639:stdout [ vrouter ] Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:1639:stdout [ mgmt-conntrd ] Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:1639 exit-code:8 exec-time:486ms queue-time:0ms Feb 17 18:39:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1666) = 0: Resource temporarily unavailable (11) Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_conntrackd after monitor op complete (interval=27000) Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_monitor_27000: master (node=node-3.test.domain.local, call=123, rc=8, cib-update=52, confirmed=false) Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: node-3.test.domain.local-p_conntrackd_monitor_27000:123 [ vrouter\n mgmt-conntrd\n ] Feb 17 18:39:53 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_conntrackd' with monitor op Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dd80 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.527 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.528 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=528 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_neutron-dhcp-agent']: word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.528 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.529 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=529 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/69, version=0.155.529) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/52) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 69 for os=Linux-3.13.0-77-generic passed Feb 17 18:39:53 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.529 to 0.155.528 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [1781] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=1781 id=04e4284e-7d18-42de-8b85-4dee13d43b78 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-1781-11) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [1781] Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1781] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1781] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1781] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3100 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3100, Current: (null), Stored: (null) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of free_swap is 3100 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: free_swap (3100) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='free_swap'] does not exist Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 71: free_swap=3100 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/71) Feb 17 18:39:53 [1781] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3100 for localhost Feb 17 18:39:53 [1781] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-1781-11) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-1781-11) state:2 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-1781-11-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-1781-11-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-1781-11-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dd80 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.529 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.530 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=530 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_conntrackd']: word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.530 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.531 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=531 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/71, version=0.155.531) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 71 for free_swap=3100 passed Feb 17 18:39:53 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.531 to 0.155.530 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [1798] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/765, version=0.155.531) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=1798 id=9cbd9963-d93f-40d8-9dce-cff3a7c65e90 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-1798-11) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [1798] Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1798] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1798] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1798] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: (null), Stored: (null) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_info is Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:39:53 [1798] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:39:53 [1798] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_info (Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_info'] does not exist Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/73) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 73: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-1798-11) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-1798-11) state:2 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-1798-11-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-1798-11-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-1798-11-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349de00 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [1801] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.531 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.532 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=532 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/73, version=0.155.532) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 73 for cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz passed Feb 17 18:39:53 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.532 to 0.155.531 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349de00 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=1801 id=07ac31ee-d99f-43be-a2ab-38800181ebaa Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-1801-11) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [1801] Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1801] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1801] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1801] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: (null), Stored: (null) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_speed is 6999.99 Feb 17 18:39:53 [1801] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:39:53 [1801] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_speed (6999.99) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/767, version=0.155.532) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_speed'] does not exist Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/75) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 75: cpu_speed=6999.99 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-1801-11) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-1801-11) state:2 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-1801-11-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-1801-11-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-1801-11-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [1815] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dec0 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/769, version=0.155.532) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=1815 id=4bc3c42c-f4fc-4fc1-8e7d-e3a195305382 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-1815-11) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [1815] Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1815] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1815] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1815] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: (null), Stored: (null) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_cores is 2 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_cores (2) Feb 17 18:39:53 [1815] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:39:53 [1815] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [1823] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dec0 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.532 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.533 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=533 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/75, version=0.155.533) Feb 17 18:39:53 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.533 to 0.155.532 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/771, version=0.155.533) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_cores'] does not exist Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 77: cpu_cores=2 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 75 for cpu_speed=6999.99 passed Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201c130 for uid=0 gid=0 pid=1823 id=ed9f3c15-89a0-4c45-9ea4-8f4dca92ff22 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-1823-12) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [1823] Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-1815-11) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-1815-11) state:2 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:39:53 [1823] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-1815-11-header Feb 17 18:39:53 [1823] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1823] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-1815-11-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-1815-11-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.64, Feb 17 18:39:53 [1823] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.64, for localhost Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.64,, Current: (null), Stored: (null) Feb 17 18:39:53 [1823] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.64, Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.64,) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/77) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] does not exist Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 79: cpu_load=0.64, Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-1823-12) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-1823-12) state:2 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-1823-12-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-1823-12-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-1823-12-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/79) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [1829] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=1829 id=01ebcba5-c0fc-44cc-b57e-e6c7d5c6f98e Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-1829-11) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [1829] Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1829] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [1829] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [1829] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df50 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.533 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.534 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=534 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/77, version=0.155.534) Feb 17 18:39:53 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.534 to 0.155.533 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 77 for cpu_cores=2 passed Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df50 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.534 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.535 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=535 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/79, version=0.155.535) Feb 17 18:39:53 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.535 to 0.155.534 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 79 for cpu_load=0.64, passed Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: (null), Stored: (null) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of ram_total is 3050 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: ram_total (3050) Feb 17 18:39:53 [1829] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:39:53 [1829] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:53 [1834] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/773, version=0.155.535) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/775, version=0.155.535) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.535 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.536 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=536 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.09 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/777, version=0.155.536) Feb 17 18:39:53 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.536 to 0.155.535 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.536 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.537 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=537 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-root_free']: @value=11683 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/779, version=0.155.537) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_total'] does not exist Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 81: ram_total=3050 Feb 17 18:39:53 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.537 to 0.155.536 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/81) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349de10 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.537 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.538 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=538 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/81, version=0.155.538) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 81 for ram_total=3050 passed Feb 17 18:39:53 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.538 to 0.155.537 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/83) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349de10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dd80 Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/781, version=0.155.538) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dd80 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/83, version=0.155.538) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.538 2 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.539 (null) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=539 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9068 Feb 17 18:39:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/783, version=0.155.539) Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:53 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] does not exist Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for root_free=(null) passed Feb 17 18:39:53 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 83 for cpu_load=0.64, passed Feb 17 18:39:53 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.539 to 0.155.538 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349de20 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: rabbit-master= Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null) Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.539 2 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.540 (null) Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=540 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='sysinfo_node-2.test.domain.local']/lrm_rsc_op[@id='sysinfo_node-2.test.domain.local_last_0']: @operation_key=sysinfo_node-2.test.domain.local_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=169:87:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;169:87:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @cal Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/367, version=0.155.540) Feb 17 18:39:54 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.540 to 0.155.539 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349de20 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] does not exist Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for var_log_free=(null) passed Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201c130 for uid=0 gid=0 pid=1834 id=bc856bd4-315c-486d-ae42-91422edf8d17 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-1834-12) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [1834] Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-1829-11) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-1829-11) state:2 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-1829-11-header Feb 17 18:39:54 [1834] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [1834] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-1829-11-header Feb 17 18:39:54 [1834] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-1829-11-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=250 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 250, Current: (null), Stored: (null) Feb 17 18:39:54 [1834] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=250 for localhost Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of ram_free is 250 Feb 17 18:39:54 [1834] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: ram_free (250) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] does not exist Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 87: ram_free=250 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-1834-12) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-1834-12) state:2 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-1834-12-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-1834-12-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-1834-12-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/87) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [1850] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a2010 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.540 2 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.541 (null) Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=541 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/87, version=0.155.541) Feb 17 18:39:54 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.541 to 0.155.540 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 87 for ram_free=250 passed Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=1850 id=376a1eef-1230-4790-9b17-2decd1d4637e Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-1850-11) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [1850] Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:54 [1850] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:54 [1850] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [1850] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11687 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11687, Current: (null), Stored: (null) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of root_free is 11687 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: root_free (11687) Feb 17 18:39:54 [1850] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11687 for localhost Feb 17 18:39:54 [1850] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a2010 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/785, version=0.155.541) Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] does not exist Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 89: root_free=11687 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-1850-11) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-1850-11) state:2 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-1850-11-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-1850-11-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-1850-11-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/89) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [1863] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dde0 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=1863 id=4ccb2c89-c768-4e4d-88ba-129807899ceb Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-1863-11) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [1863] Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [1863] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [1863] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [1863] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11687 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11687, Current: 11687, Stored: (null) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of root_free is 11687 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: root_free (11687) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:39:54 [1863] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11687 for localhost Feb 17 18:39:54 [1863] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.541 2 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.542 (null) Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=542 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/89, version=0.155.542) Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 91: root_free=11687 Feb 17 18:39:54 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.542 to 0.155.541 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 89 for root_free=11687 passed Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-1863-11) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-1863-11) state:2 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-1863-11-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-1863-11-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-1863-11-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/91) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dde0 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349ded0 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/787, version=0.155.542) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [1876] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=1876 id=2f2aeccf-9f1c-4813-959e-837f6afeb200 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-1876-11) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [1876] Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/91, version=0.155.542) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 91 for root_free=11687 passed Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [1876] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [1876] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [1876] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9072 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9072, Current: (null), Stored: (null) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9072 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9072) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [1876] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9072 for localhost Feb 17 18:39:54 [1876] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349ded0 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/789, version=0.155.542) Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] does not exist Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 93: var_log_free=9072 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/93) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-1876-11) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-1876-11) state:2 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-1876-11-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-1876-11-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-1876-11-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349db20 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [1892] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.542 2 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.543 (null) Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=543 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/93, version=0.155.543) Feb 17 18:39:54 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.543 to 0.155.542 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 93 for var_log_free=9072 passed Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=1892 id=52456fd2-4dd0-476c-ab06-f7f4d2a5441c Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-1892-11) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [1892] Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [1892] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [1892] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [1892] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18422 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18422, Current: (null), Stored: (null) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_lib_mysql_free is 18422 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_lib_mysql_free (18422) Feb 17 18:39:54 [1892] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18422 for localhost Feb 17 18:39:54 [1892] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1424) = 0: Resource temporarily unavailable (11) Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1449) = 0: Resource temporarily unavailable (11) Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:1666 - exited with rc=0 Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:1666:stderr [ -- empty -- ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:1666:stdout [ arch: x86_64 ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:1666:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:1666:stdout [ free_swap: 3100 ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:1666:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:1666:stdout [ cpu_speed: 6999.99 ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:1666:stdout [ cpu_cores: 2 ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:1666:stdout [ cpu_load: 0.64, ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:1666:stdout [ ram_total: 3050 ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:1666:stdout [ ram_free: 250 ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:1666:stdout [ root_free: 11687 ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:1666:stdout [ root_free: 11687 ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:1666:stdout [ var_log_free: 9072 ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:1666:stdout [ var_lib_mysql_free: 18422 ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:start call_id:126 pid:1666 exit-code:0 exec-time:1056ms queue-time:2ms Feb 17 18:39:54 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource sysinfo_node-3.test.domain.local after start op complete (interval=0) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349db20 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/791, version=0.155.543) Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_lib_mysql_free'] does not exist Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 95: var_lib_mysql_free=18422 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/95) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-1892-11) Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-1892-11) state:2 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-1892-11-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-1892-11-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-1892-11-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed SysInfo_meta-data_0 process 1893 exited with rc=0 Feb 17 18:39:54 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation sysinfo_node-3.test.domain.local_start_0: ok (node=node-3.test.domain.local, call=126, rc=0, cib-update=53, confirmed=true) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6707] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-sysinfo_node-3.test.domain.local_start_0:126 [ arch: x86_64\nos: Linux-3.13.0-77-generic\nfree_swap: 3100\ncpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz\ncpu_speed: 6999.99\ncpu_cores: 2\ncpu_load: 0.64,\nram_total: 3050\nram_free: 250\nroot_free: 11687\nroot_free: 11687\nvar_log_free: 9072\nvar_lib_mysql_free: 18422\n ] Feb 17 18:39:54 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'sysinfo_node-3.test.domain.local' with start op Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e3a0 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.543 2 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.544 (null) Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=544 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/95, version=0.155.544) Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/53) Feb 17 18:39:54 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.544 to 0.155.543 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:54 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 95 for var_lib_mysql_free=18422 passed Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e3a0 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/793, version=0.155.544) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.544 2 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.545 (null) Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=545 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='sysinfo_node-3.test.domain.local']/lrm_rsc_op[@id='sysinfo_node-3.test.domain.local_last_0']: @operation_key=sysinfo_node-3.test.domain.local_start_0, @operation=start, @transition-key=167:87:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;167:87:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=126, @rc-code=0, @last-run=145573 Feb 17 18:39:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/53, version=0.155.545) Feb 17 18:39:54 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.545 to 0.155.544 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_IPaddr2: 2016/02/17_18:39:54 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.743ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 6.261ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.541ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.546ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:1424 - exited with rc=0 Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:1424:stderr [ -- empty -- ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:1424:stdout [ -- empty -- ] Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:1424 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(1449) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:39:55 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.632ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.532ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.534ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:39:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:1449 - exited with rc=0 Feb 17 18:39:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:1449:stderr [ -- empty -- ] Feb 17 18:39:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:1449:stdout [ -- empty -- ] Feb 17 18:39:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:1449 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:55 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:55 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:55 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:39:55 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:39:55 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:55 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:55 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:55 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:39:55 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.545 2 Feb 17 18:39:55 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.546 (null) Feb 17 18:39:55 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=546 Feb 17 18:39:55 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='ping_vip__public']/lrm_rsc_op[@id='ping_vip__public_last_0']: @operation_key=ping_vip__public_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=171:87:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;171:87:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=277, @last-run=1455734393, @last-rc-change= Feb 17 18:39:55 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/368, version=0.155.546) Feb 17 18:39:55 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.546 to 0.155.545 Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:39:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:39:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:39:58 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:39:58 INFO: net.ipv4.ip_forward = 1 Feb 17 18:39:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:2168 - exited with rc=0 Feb 17 18:39:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:2168:stderr [ -- empty -- ] Feb 17 18:39:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:2168:stdout [ -- empty -- ] Feb 17 18:39:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:2168 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:39:59 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:40:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:00 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:00 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:00 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:00 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:00 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:00 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:00 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:00 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:00 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 8363ec3878fc95476f1cbbe80df89d9c for 0.155.546 (0x1733cf0 0) Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:00 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:00 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:00 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:00 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:00 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:00 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:00 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:00 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.546 2 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.547 (null) Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=547 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_mysql']/lrm_rsc_op[@id='p_mysql_last_0']: @operation_key=p_mysql_stop_0, @operation=stop, @transition-key=6:87:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;6:87:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=274, @last-run=1455734393, @last-rc-change=1455734393, @exec-time=9440 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/369, version=0.155.547) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:02 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.547 to 0.155.546 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=168:88:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=127, reply=1, notify=0, exit=4202464 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_mysql_stop_0 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=16:88:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_mysql_stop_0 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=128, reply=1, notify=0, exit=4202464 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_mysql action:stop call_id:128 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [2425] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=2425 id=bec2a2ad-033e-4391-b80d-f6e28c0fd0f5 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-2425-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [2425] Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2425] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2425] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2425] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:40:02 [2425] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:40:02 [2425] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-2425-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-2425-11) state:2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-2425-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-2425-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-2425-11-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=235:88:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_rabbitmq-server_notify_0 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=129, reply=1, notify=0, exit=4202464 Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_rabbitmq-server action:notify call_id:129 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_neutron-metadata-agent_start_0 Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=160:88:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-metadata-agent_start_0 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=130, reply=1, notify=0, exit=4202464 Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_neutron-metadata-agent action:start call_id:130 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:02 [2432] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=2432 id=3a416487-28b7-46f9-b3ec-900dc2e5d557 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-2432-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [2432] Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2432] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2432] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2432] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:40:02 [2432] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:40:02 [2432] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-2432-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-2432-11) state:2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-2432-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-2432-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-2432-11-header Feb 17 18:40:02 [2459] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=2459 id=adc29495-03fd-43b4-96c2-8062e950f5e6 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-2459-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [2459] Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2459] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2459] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2459] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3100 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3100, Current: 3100, Stored: 3100 Feb 17 18:40:02 [2459] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3100 for localhost Feb 17 18:40:02 [2459] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-2459-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-2459-11) state:2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-2459-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-2459-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-2459-11-header Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:40:02 [2495] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=2495 id=af24105d-15b1-4dd1-81fb-a2f542af9bd4 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-2495-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [2495] Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2495] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2495] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2495] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:40:02 [2495] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:40:02 [2495] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-2495-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-2495-11) state:2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-2495-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-2495-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-2495-11-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-2486-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [2486] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761349cf10, cpd=0x7f761349f4f4 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f761349cf10 Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-6268-2486-26-header Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-6268-2486-26-header Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-6268-2486-26-header Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-2486-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-2486-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-2486-26-header Feb 17 18:40:02 [2499] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-2486-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-2486-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-2486-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [2486] Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=2499 id=7434ce8b-9b38-4b67-af40-47c5f42a8f8b Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-2499-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [2499] Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2499] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2499] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2499] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:40:02 [2499] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:40:02 [2499] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-2499-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-2499-11) state:2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-2499-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-2499-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-2499-11-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-2486-26-header Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-2486-26-header Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-2486-26-header Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:40:02 [2486] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-2486-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-2486-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-2486-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-2486-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-2486-26-header Feb 17 18:40:02 [2504] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=2504 id=ab6e3289-90df-47f2-bb6c-8fbd5917a3cc Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-2504-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [2504] Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2504] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2504] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2504] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:40:02 [2504] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:40:02 [2504] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-2504-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-2504-11) state:2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-2504-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-2504-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-2504-11-header Feb 17 18:40:02 [2521] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=2521 id=6ffbeb28-f4c4-47ac-b37f-393c9195dd6d Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-2521-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [2521] Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2521] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2521] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2521] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.67, Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.67,, Current: 0.64,, Stored: 0.64, Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.67, Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.67,) Feb 17 18:40:02 [2521] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.67, for localhost Feb 17 18:40:02 [2521] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 97: cpu_load=0.67, Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-2521-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-2521-11) state:2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-2521-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-2521-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-2521-11-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/97) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-neutron-metadata-agent: 2016/02/17_18:40:02 WARNING: OpenStack Neutron agent 'neutron-metadata-agent' not running. Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.547 2 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.548 (null) Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=548 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.67, Feb 17 18:40:02 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.548 to 0.155.547 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/97, version=0.155.548) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 97 for cpu_load=0.67, passed Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-neutron-metadata-agent: 2016/02/17_18:40:02 DEBUG: Create pid file: /var/run/resource-agents/ocf-neutron-metadata-agent/ocf-neutron-metadata-agent.pid with content Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [2537] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=2537 id=d5ee7473-1840-427f-9052-29672e9819ff Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-2537-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [2537] Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.548 2 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.549 (null) Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=549 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_ntp']/lrm_rsc_op[@id='p_ntp_last_0']: @operation_key=p_ntp_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=181:88:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;181:88:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=281, @last-run=1455734402, @last-rc-change=1455734402, @exec-time=133 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/371, version=0.155.549) Feb 17 18:40:02 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.549 to 0.155.548 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-neutron-metadata-agent: 2016/02/17_18:40:02 WARNING: OpenStack Neutron agent (neutron-metadata-agent) was run, but no PID file found. Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:02 [2537] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2537] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2537] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [2537] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:40:02 [2537] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-2537-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-2537-11) state:2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-2537-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-2537-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-2537-11-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-neutron-metadata-agent: 2016/02/17_18:40:02 WARNING: Writing PID='2541' to '/var/run/resource-agents/ocf-neutron-metadata-agent/ocf-neutron-metadata-agent.pid' for 'neutron-metadata-agent' worker... ocf-neutron-metadata-agent: 2016/02/17_18:40:02 INFO: OpenStack Neutron Metadata Agent (neutron-metadata-agent) started Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2286) = 0: Resource temporarily unavailable (11) Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2339) = 0: Resource temporarily unavailable (11) Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2417) = 0: Resource temporarily unavailable (11) Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2426) = 0: Resource temporarily unavailable (11) Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2431) = 0: Resource temporarily unavailable (11) Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_start_0:2433 - exited with rc=0 Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_neutron-metadata-agent_start_0:2433:stderr [ sh: 1: cannot create : Directory nonexistent ] Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_neutron-metadata-agent_start_0:2433:stderr [ cat: /var/run/resource-agents/ocf-neutron-metadata-agent/ocf-neutron-metadata-agent.pid: No such file or directory ] Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_start_0:2433:stdout [ -- empty -- ] Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_neutron-metadata-agent action:start call_id:130 pid:2433 exit-code:0 exec-time:178ms queue-time:0ms Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-metadata-agent after start op complete (interval=0) Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/795, version=0.155.549) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ocf-neutron-metadata-agent_meta-data_0 process 2584 exited with rc=0 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/54) Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-metadata-agent_start_0: ok (node=node-3.test.domain.local, call=130, rc=0, cib-update=54, confirmed=true) Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_neutron-metadata-agent_start_0:130 [ sh: 1: cannot create : Directory nonexistent\ncat: /var/run/resource-agents/ocf-neutron-metadata-agent/ocf-neutron-metadata-agent.pid: No such file or directory\n ] Feb 17 18:40:02 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-metadata-agent' with start op Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.549 2 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.550 (null) Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=550 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_neutron-metadata-agent']/lrm_rsc_op[@id='p_neutron-metadata-agent_last_0']: @operation_key=p_neutron-metadata-agent_start_0, @operation=start, @transition-key=160:88:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;160:88:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=130, @rc-code=0, @last-run=1455734402, @last-rc-change=14 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/54, version=0.155.550) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:02 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.550 to 0.155.549 ocf-ns_IPaddr2: 2016/02/17_18:40:02 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.606ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.630ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.527ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.542ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:2286 - exited with rc=0 Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:2286:stderr [ -- empty -- ] Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:2286:stdout [ -- empty -- ] Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:2286 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2339) = 0: Resource temporarily unavailable (11) Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2417) = 0: Resource temporarily unavailable (11) Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2426) = 0: Resource temporarily unavailable (11) Feb 17 18:40:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2431) = 0: Resource temporarily unavailable (11) Feb 17 18:40:02 [2597] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=2597 id=1e87377a-61f6-4ae5-914a-1693ae425ba0 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-2597-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [2597] Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2597] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2597] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2597] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=250 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 250, Current: 250, Stored: 250 Feb 17 18:40:02 [2597] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=250 for localhost Feb 17 18:40:02 [2597] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-2597-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-2597-11) state:2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-2597-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-2597-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-2597-11-header Feb 17 18:40:02 [2621] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=2621 id=5c541491-af6e-4b29-b61b-89b6999a8ac6 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-2621-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [2621] Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2621] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2621] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [2621] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11686 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11686, Current: 11687, Stored: 11687 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of root_free is 11686 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: root_free (11686) Feb 17 18:40:02 [2621] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11686 for localhost Feb 17 18:40:02 [2621] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.550 2 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.551 (null) Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=551 Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='sysinfo_node-2.test.domain.local']: Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 99: root_free=11686 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-2621-11) Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-2621-11) state:2 Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-2621-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-2621-11-header Feb 17 18:40:02 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-2621-11-header Feb 17 18:40:02 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/99) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:02 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [2643] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349de70 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=2643 id=268012cb-cdfc-42a0-bede-3cf4f70eeae8 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-2643-11) Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [2643] Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:03 [2643] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:03 [2643] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [2643] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11686 Feb 17 18:40:03 [2643] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11686 for localhost Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11686, Current: 11686, Stored: 11687 Feb 17 18:40:03 [2643] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of root_free is 11686 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: root_free (11686) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.551 2 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.552 (null) Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=552 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-root_free']: @value=11686 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/99, version=0.155.552) Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:03 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.552 to 0.155.551 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 101: root_free=11686 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 99 for root_free=11686 passed Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-2643-11) Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-2643-11) state:2 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-2643-11-header Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-2643-11-header Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-2643-11-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/101) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349de70 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [2673] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 lrmd: 2016/02/17_18:40:03 INFO: p_rabbitmq-server: notify: pre-promote begin. Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created lrmd: 2016/02/17_18:40:03 INFO: p_rabbitmq-server: my_host(): hostlist is: node-2.test.domain.local Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349de70 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: rabbit-master= Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 lrmd: 2016/02/17_18:40:03 DEBUG: p_rabbitmq-server: my_host(): comparing 'messaging-node-3' with 'messaging-node-2' Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2339) = 0: Resource temporarily unavailable (11) Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2417) = 0: Resource temporarily unavailable (11) Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2426) = 0: Resource temporarily unavailable (11) Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_notify_0:2431 - exited with rc=0 Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:2431:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:2431:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:2431:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:2431:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:2431:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:2431:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_notify_0:2431:stdout [ -- empty -- ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_rabbitmq-server action:notify call_id:129 pid:2431 exit-code:0 exec-time:317ms queue-time:0ms Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-28) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a24f0 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/797, version=0.155.552) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349de70 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-28-header Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-28-header Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-28-header Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_rabbitmq-server after notify op complete (interval=0) Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=2673 id=305a7e95-070f-431f-85e4-a58b7d9a84f1 Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_rabbitmq-server_notify_0 from 235:88:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455734403-18 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-2673-11) Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [2673] Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_rabbitmq-server_notify_0: ok (node=node-3.test.domain.local, call=129, rc=0, cib-update=0, confirmed=true) Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_rabbitmq-server_notify_0:129 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n ] Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [2673] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [2673] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [2673] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9070 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9070, Current: 9072, Stored: 9072 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9070 Feb 17 18:40:03 [2673] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9070 for localhost Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9070) Feb 17 18:40:03 [2673] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349de70 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-28) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-28) state:2 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a24f0 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-28-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-28-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-28-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/101, version=0.155.552) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349de70 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349de70 Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/799, version=0.155.552) Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 103: var_log_free=9070 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 101 for root_free=11686 passed Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-2673-11) Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-2673-11) state:2 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-2673-11-header Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-2673-11-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/103) Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-2673-11-header Feb 17 18:40:03 [2704] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=2704 id=b100136a-199c-43be-81c7-39ae78d55ab7 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-2704-11) Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [2704] Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [2704] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [2704] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [2704] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18422 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18422, Current: 18422, Stored: 18422 Feb 17 18:40:03 [2704] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18422 for localhost Feb 17 18:40:03 [2704] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-2704-11) Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-2704-11) state:2 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-2704-11-header Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-2704-11-header Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-2704-11-header Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2339) = 0: Resource temporarily unavailable (11) Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:2417 - exited with rc=0 Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:2417:stderr [ -- empty -- ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:2417:stdout [ arch: x86_64 ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:2417:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:2417:stdout [ free_swap: 3100 ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:2417:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:2417:stdout [ cpu_speed: 6999.99 ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:2417:stdout [ cpu_cores: 2 ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:2417:stdout [ cpu_load: 0.67, ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:2417:stdout [ ram_total: 3050 ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:2417:stdout [ ram_free: 250 ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:2417:stdout [ root_free: 11686 ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:2417:stdout [ root_free: 11686 ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:2417:stdout [ var_log_free: 9070 ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:2417:stdout [ var_lib_mysql_free: 18422 ] Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:2417 exit-code:0 exec-time:372ms queue-time:0ms Feb 17 18:40:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2426) = 0: Resource temporarily unavailable (11) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349de70 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource sysinfo_node-3.test.domain.local after monitor op complete (interval=15000) Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation sysinfo_node-3.test.domain.local_monitor_15000: ok (node=node-3.test.domain.local, call=127, rc=0, cib-update=55, confirmed=false) Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-sysinfo_node-3.test.domain.local_monitor_15000:127 [ arch: x86_64\nos: Linux-3.13.0-77-generic\nfree_swap: 3100\ncpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz\ncpu_speed: 6999.99\ncpu_cores: 2\ncpu_load: 0.67,\nram_total: 3050\nram_free: 250\nroot_free: 11686\nroot_free: 11686\nvar_log_free: 9070\nvar_lib_mysql_free: 18422\n ] Feb 17 18:40:03 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'sysinfo_node-3.test.domain.local' with monitor op Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349de70 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.552 2 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.553 (null) Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=553 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9070 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/103, version=0.155.553) Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/55) Feb 17 18:40:03 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.553 to 0.155.552 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349de70 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349de70 Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:03 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 103 for var_log_free=9070 passed Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/801, version=0.155.553) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349de70 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:03 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:03 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.553 2 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.554 (null) Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=554 Feb 17 18:40:03 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='sysinfo_node-3.test.domain.local']: word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.555 2 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.556 (null) Feb 17 18:40:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=556 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_mysql']/lrm_rsc_op[@id='p_mysql_last_0']: @operation_key=p_mysql_stop_0, @operation=stop, @transition-key=16:88:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;16:88:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=128, @rc-code=0, @op-status=0, @last-run=1455734402, @last-rc-change=1455734402, @exec-time=5104 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/56, version=0.155.556) Feb 17 18:40:07 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.556 to 0.155.555 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=164:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-metadata-agent_monitor_60000 Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=131, reply=1, notify=0, exit=4202464 Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_neutron-metadata-agent action:monitor call_id:131 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_mysql_start_0 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=82:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_mysql_start_0 Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=132, reply=1, notify=0, exit=4202464 Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_mysql action:start call_id:132 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=236:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_rabbitmq-server_notify_0 Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=133, reply=1, notify=0, exit=4202464 Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_rabbitmq-server action:notify call_id:133 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for ping_vip__public_start_0 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=175:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=ping_vip__public_start_0 Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=134, reply=1, notify=0, exit=4202464 Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:ping_vip__public action:start call_id:134 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.556 2 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.557 (null) Feb 17 18:40:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=557 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_ntp']: word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761349cf10, cpd=0x7f761349d7f4 Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f761349cf10 Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-6268-2972-26-header Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-6268-2972-26-header Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-6268-2972-26-header Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-2972-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-2972-26) state:2 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-2972-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-2972-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-2972-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-2972-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [2972] Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:2908 - exited with rc=0 Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:2908:stderr [ -- empty -- ] Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:2908:stdout [ -- empty -- ] Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-metadata-agent action:monitor call_id:131 pid:2908 exit-code:0 exec-time:90ms queue-time:0ms Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2914) = 0: Resource temporarily unavailable (11) Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2918) = 0: Resource temporarily unavailable (11) Feb 17 18:40:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2976) = 0: Resource temporarily unavailable (11) Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-metadata-agent after monitor op complete (interval=60000) Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-2972-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-2972-26) state:2 Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-metadata-agent_monitor_60000: ok (node=node-3.test.domain.local, call=131, rc=0, cib-update=57, confirmed=false) Feb 17 18:40:07 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-metadata-agent' with monitor op Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-2972-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-2972-26-header Feb 17 18:40:07 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/57) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-2972-26-header Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-2972-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-2972-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-2972-26-header Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:40:07 [2972] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:07 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.557 2 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.558 (null) Feb 17 18:40:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=558 Feb 17 18:40:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_neutron-metadata-agent']: word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df90 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.560 2 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.561 (null) Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=561 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-master-p_rabbitmq-server']: @value=1000 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/803, version=0.155.561) Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_rabbitmq-server'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[3]) Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:09 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.561 to 0.155.560 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df90 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/105) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/105, version=0.155.561) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 105 for master-p_rabbitmq-server=1 passed Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349db40 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.561 2 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.562 (null) Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=562 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/805, version=0.155.562) Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit-master'] does not exist Feb 17 18:40:09 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.562 to 0.155.561 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for rabbit-master=(null) passed Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349db40 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 Feb 17 18:40:09 [3283] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=3283 id=9cdcef9b-3b7c-41da-bb69-8ded26699b78 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-3283-11) Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [3283] Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:09 [3283] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:09 [3283] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:09 [3283] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: pingd=1000 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for pingd Feb 17 18:40:09 [3283] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: pingd=1000 for localhost Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1000, Current: (null), Stored: (null) Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of pingd is 1000 Feb 17 18:40:09 [3283] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-3283-11) Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-3283-11) state:2 Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-3283-11-header Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-3283-11-header Feb 17 18:40:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-3283-11-header Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_start_0:2918 - exited with rc=0 Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_start_0:2918:stderr [ -- empty -- ] Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_start_0:2918:stdout [ -- empty -- ] Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:ping_vip__public action:start call_id:134 pid:2918 exit-code:0 exec-time:2041ms queue-time:0ms Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2976) = 0: Resource temporarily unavailable (11) Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(3107) = 0: Resource temporarily unavailable (11) Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(3272) = 0: Resource temporarily unavailable (11) Feb 17 18:40:09 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource ping_vip__public after start op complete (interval=0) Feb 17 18:40:09 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ping_meta-data_0 process 3284 exited with rc=0 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/60) Feb 17 18:40:09 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation ping_vip__public_start_0: ok (node=node-3.test.domain.local, call=134, rc=0, cib-update=60, confirmed=true) Feb 17 18:40:09 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'ping_vip__public' with start op Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:09 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.562 2 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.563 (null) Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=563 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='ping_vip__public']/lrm_rsc_op[@id='ping_vip__public_last_0']: @operation_key=ping_vip__public_start_0, @operation=start, @transition-key=175:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;175:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=134, @rc-code=0, @last-run=1455734407, @last-rc-change=1455734407, @exec-time=204 Feb 17 18:40:09 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/60, version=0.155.563) Feb 17 18:40:09 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.563 to 0.155.562 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_dns: 2016/02/17_18:40:09 INFO: dnsmasq daemon running Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2976) = 0: Resource temporarily unavailable (11) Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(3107) = 0: Resource temporarily unavailable (11) Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:3272 - exited with rc=0 Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:3272:stderr [ -- empty -- ] Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:3272:stdout [ -- empty -- ] Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:3272 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:40:10 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2976) = 0: Resource temporarily unavailable (11) Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(3107) = 0: Resource temporarily unavailable (11) Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:3294 - exited with rc=0 Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:3294:stderr [ -- empty -- ] Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:3294:stdout [ -- empty -- ] Feb 17 18:40:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:3294 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:40:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2976) = 0: Resource temporarily unavailable (11) Feb 17 18:40:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(3107) = 0: Resource temporarily unavailable (11) Feb 17 18:40:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:3363 - exited with rc=0 Feb 17 18:40:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:3363:stderr [ -- empty -- ] Feb 17 18:40:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:3363:stdout [ -- empty -- ] Feb 17 18:40:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:3363 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:40:11 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.722ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.742ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.543ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.542ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:40:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:40:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:2976 - exited with rc=0 Feb 17 18:40:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:2976:stderr [ -- empty -- ] Feb 17 18:40:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:2976:stdout [ -- empty -- ] Feb 17 18:40:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:2976 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(3107) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:40:11 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.671ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.528ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.525ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:40:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:3107 - exited with rc=0 Feb 17 18:40:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:3107:stderr [ -- empty -- ] Feb 17 18:40:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:3107:stdout [ -- empty -- ] Feb 17 18:40:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:3107 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:40:12 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:40:12 INFO: MySQL is not running Feb 17 18:40:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:40:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 Feb 17 18:40:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:3781 - exited with rc=0 Feb 17 18:40:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:3781:stderr [ -- empty -- ] Feb 17 18:40:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:3781:stdout [ -- empty -- ] Feb 17 18:40:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:3781 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(3805) = 0: Resource temporarily unavailable (11) ocf-neutron-dhcp-agent: 2016/02/17_18:40:13 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:40:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:3805 - exited with rc=0 Feb 17 18:40:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:3805:stderr [ -- empty -- ] Feb 17 18:40:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:3805:stdout [ -- empty -- ] Feb 17 18:40:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:3805 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dcd0 Feb 17 18:40:13 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:13 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:13 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:13 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:13 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.563 2 Feb 17 18:40:13 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.564 (null) Feb 17 18:40:13 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=564 Feb 17 18:40:13 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-master-p_rabbitmq-server']: @value=1 Feb 17 18:40:13 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/807, version=0.155.564) Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_rabbitmq-server'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[3]) Feb 17 18:40:13 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.564 to 0.155.563 Feb 17 18:40:13 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:13 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/108) Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dcd0 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:13 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:13 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/108, version=0.155.564) Feb 17 18:40:13 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 108 for master-p_rabbitmq-server=1 passed Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:13 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:40:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:40:17 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:40:18 ERROR: MySQL is not running ocf-mysql-wss: 2016/02/17_18:40:18 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1 ocf-mysql-wss: 2016/02/17_18:40:18 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=4162 id=4acca0df-cbe5-4db3-8d58-fafc59e04e07 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-4162-14) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [4162] Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=4162 id=69e60803-52b1-4acd-a66e-995701ec732f Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-4162-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [4162] Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1, Current: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1, Stored: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:-1 for node-3.test.domain.local Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: info: main: Update gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:-1 sent via attrd Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-4162-14) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-4162-14) state:2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4162-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4162-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4162-14-header Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4162-14-header Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4162-14-header Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4162-14-header Feb 17 18:40:18 [4162] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-4162-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-4162-11) state:2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-4162-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-4162-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-4162-11-header Feb 17 18:40:18 [4164] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:40:18 [4164] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:40:18 [4164] node-3.test.domain.local crm_node: debug: main: Attempting to process -q command for cluster type: corosync Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-4164-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [4164] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:18 [4164] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [4164] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [4164] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7f761349cf10 Feb 17 18:40:18 [4164] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-4164-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-4164-26) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-6268-4164-26-header Feb 17 18:40:18 [4164] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-request-6268-4164-26-header Feb 17 18:40:18 [4164] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-response-6268-4164-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-6268-4164-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-6268-4164-26-header Feb 17 18:40:18 [4164] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-event-6268-4164-26-header Feb 17 18:40:18 [4164] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [4173] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:40:18 [4173] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:40:18 [4173] node-3.test.domain.local crm_node: debug: main: Attempting to process -l command for cluster type: corosync Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=4173 id=5846dd77-528e-4cc6-b184-d0300082fcf8 Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-4173-12) Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [4173] Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4173] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4173] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4173] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4173] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-4173-12) Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-4173-12) state:2 Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-4173-12-header Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-4173-12-header Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-4173-12-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=4177 id=ed2e3873-8718-4f90-bdbe-3a7778c9cb29 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-4177-14) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [4177] Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: unpack_config: Stop all active resources: false Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: unpack_config: Default stickiness: 0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: determine_op_status: p_mysql_monitor_60000 on node-2.test.domain.local returned 'unknown error' (1) instead of the expected value: 'ok' (0) Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:0 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:0 on node-3.test.domain.local: unknown error (1) Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:1 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:1 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-3.test.domain.local to ping_vip__public:1 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-3.test.domain.local to p_ntp:1 Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-4177-14) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-4177-14) state:2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4177-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4177-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4177-14-header Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4177-14-header Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4177-14-header Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4177-14-header Feb 17 18:40:18 [4177] node-3.test.domain.local crm_resource: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=4182 id=32c6fc53-7651-4d16-9948-d5e31573d27f Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-4182-14) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [4182] Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4182] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4182] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4182] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4182] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/configuration/crm_config//cluster_property_set//nvpair[@name='galera_master_timeout'] does not exist Feb 17 18:40:18 [4182] node-3.test.domain.local crm_attribute: info: main: Read galera_master_timeout=300 Feb 17 18:40:18 [4182] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:40:18 [4182] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-4182-14) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-4182-14) state:2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4182-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4182-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4182-14-header Feb 17 18:40:18 [4182] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4182-14-header Feb 17 18:40:18 [4182] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4182-14-header Feb 17 18:40:18 [4182] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4182-14-header Feb 17 18:40:18 [4182] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 ocf-mysql-wss: 2016/02/17_18:40:18 INFO: Checking if Primary Component Feb 17 18:40:18 [4192] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:40:18 [4192] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:40:18 [4192] node-3.test.domain.local crm_node: debug: main: Attempting to process -p command for cluster type: corosync Feb 17 18:40:18 [4200] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=4192 id=c9d24f5f-fc13-4b38-bb90-a09baddd6b18 Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-4192-12) Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [4192] Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4192] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4192] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4192] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4192] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-4192-12) Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-4192-12) state:2 Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-4192-12-header Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-4192-12-header Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-4192-12-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=4200 id=8c193d09-6268-431e-a55b-9645166a659a Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-4200-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [4200] Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4200] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4200] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4200] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:40:18 [4200] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:40:18 [4200] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-4200-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-4200-11) state:2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-4200-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-4200-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-4200-11-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:18 [4208] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dd50 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.564 2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.565 (null) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=565 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.10 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/809, version=0.155.565) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=4204 id=85651fb0-5bb8-4f03-80a5-cb667bff893f Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-4204-14) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [4204] Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.565 to 0.155.564 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=4208 id=5e4d0874-47b0-482d-996e-fa3640873cbf Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-4208-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [4208] Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/110) Feb 17 18:40:18 [4208] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4208] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4208] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:40:18 [4208] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:40:18 [4208] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-4208-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-4208-11) state:2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-4208-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-4208-11-header Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-4208-11-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-2.test.domain.local to 2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='2']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: info: main: Read gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:20770 Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-4204-14) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-4204-14) state:2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4204-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4204-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4204-14-header Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4204-14-header Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4204-14-header Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4204-14-header Feb 17 18:40:18 [4204] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dd50 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/110, version=0.155.565) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 110 for cpu_load=0.67, passed Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-mysql-wss: 2016/02/17_18:40:18 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:20770 Feb 17 18:40:18 [4233] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining ocf-mysql-wss: 2016/02/17_18:40:18 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:20770 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=4233 id=b30ae4b8-bb2d-4a0c-9b25-62cfca3d01a2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-4233-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [4233] Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4233] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4233] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4233] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3100 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3100, Current: 3100, Stored: 3100 Feb 17 18:40:18 [4233] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3100 for localhost Feb 17 18:40:18 [4233] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-4233-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-4233-11) state:2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-4233-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-4233-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-4233-11-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=4245 id=a730fb90-e47c-46c4-9a61-29d1a13b75dd Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-4245-14) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [4245] Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4252] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[7]) Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: info: main: Read gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4245-14-header Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4245-14-header Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4245-14-header Feb 17 18:40:18 [4245] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-4245-14) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-4245-14) state:2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4245-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4245-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4245-14-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=4252 id=a24ed48f-2be5-4016-9c71-11ac7bd78406 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-4252-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [4252] Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 ocf-mysql-wss: 2016/02/17_18:40:18 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4252] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4252] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4252] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:40:18 [4252] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:40:18 [4252] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-4252-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-4252-11) state:2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-4252-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-4252-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-4252-11-header ocf-mysql-wss: 2016/02/17_18:40:18 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:40:18 [4261] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=4261 id=815c6825-37bf-449b-a17d-b98e0fd47744 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-4261-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [4261] Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4261] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4261] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4261] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:40:18 [4261] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:40:18 [4261] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-4261-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-4261-11) state:2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-4261-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-4261-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-4261-11-header Feb 17 18:40:18 [4272] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=4272 id=b2f32b93-3e8a-4b89-bb6f-1880fa497ccd Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-4272-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [4272] Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4272] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4272] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4272] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:40:18 [4272] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:40:18 [4272] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-4272-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-4272-11) state:2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-4272-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-4272-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-4272-11-header Feb 17 18:40:18 [4283] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=4275 id=cc0fc864-d387-4047-a1f7-e9e6243838be Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-4275-14) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [4275] Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=4283 id=2bdec5c9-dc75-4daf-b198-d826efeb6f39 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-4283-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [4283] Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4283] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4283] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4283] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.68, Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.68,, Current: 0.67,, Stored: 0.67, Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.68, Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.68,) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4275-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-4275-14) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4275-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-4275-14) state:2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4275-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4275-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4275-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4275-14-header Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: validate_with_relaxng: Creating RNG parser context Feb 17 18:40:18 [4283] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.68, for localhost Feb 17 18:40:18 [4283] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/112) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 112: cpu_load=0.68, Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-4283-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-4283-11) state:2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-4283-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-4283-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-4283-11-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349eaa0 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.565 2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.566 (null) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=566 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.68, Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/112, version=0.155.566) Feb 17 18:40:18 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.566 to 0.155.565 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 112 for cpu_load=0.68, passed Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349eaa0 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:18 [4288] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=4288 id=a49ef766-7ccb-4e4e-9b07-2d9e1a8e69ac Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-4288-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [4288] Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4288] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4288] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4288] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:40:18 [4288] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:40:18 [4288] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-4288-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-4288-11) state:2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-4288-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-4288-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-4288-11-header Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: cib_file_signon: crm_simulate: Opened connection to local file '/var/lib/pacemaker/cib/shadow.4275' Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: cib_file_perform_op_delegate: cib_query on (null) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: unpack_config: Stop all active resources: false Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: unpack_config: Default stickiness: 0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:40:18 [4293] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_mysql_monitor_60000 on node-2.test.domain.local returned 'unknown error' (1) instead of the expected value: 'ok' (0) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:0 on node-3.test.domain.local: unknown error (1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-3.test.domain.local to ping_vip__public:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-3.test.domain.local to p_ntp:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/811, version=0.155.566) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: apply_system_health: Applying automated node health strategy: migrate-on-red Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-1.test.domain.local has an combined system health of 0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-2.test.domain.local has an combined system health of 0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-3.test.domain.local has an combined system health of 0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-1.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_vrouter [p_vrouter] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_print: vip__management (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter_pub (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_print: vip__public (ocf::fuel:ns_IPaddr2): Stopped Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_conntrackd [p_conntrackd] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: short_print: Masters: [ node-3.test.domain.local ] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: short_print: Slaves: [ node-2.test.domain.local ] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_haproxy [p_haproxy] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_mysql [p_mysql] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: short_print: Slaves: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_dns [p_dns] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_heat-engine [p_heat-engine] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-plugin-openvswitch-agent [p_neutron-plugin-openvswitch-agent] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-l3-agent [p_neutron-l3-agent] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-dhcp-agent [p_neutron-dhcp-agent] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-metadata-agent [p_neutron-metadata-agent] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-3.test.domain.local (ocf::pacemaker:SysInfo): Started node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-2.test.domain.local (ocf::pacemaker:SysInfo): Started node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_ping_vip__public [ping_vip__public] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_ntp [p_ntp] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__management: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 38 times on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 38 times on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 38 times on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_rabbitmq-server:0: preferring current location (node=node-2.test.domain.local, weight=100) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_heat-engine:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-plugin-openvswitch-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-l3-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:0 has failed 4 times on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:1 has failed 4 times on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-dhcp-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-metadata-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource ping_vip__public:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_ntp:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter_pub: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 4 times on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999996 more times on node-3.test.domain.local before being forced off Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 4 times on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999996 more times on node-3.test.domain.local before being forced off Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 4 times on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999996 more times on node-3.test.domain.local before being forced off Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_rabbitmq-server:1: preferring current location (node=node-3.test.domain.local, weight=100) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_heat-engine:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-plugin-openvswitch-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-l3-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-dhcp-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-metadata-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource ping_vip__public:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_ntp:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=4293 id=f5510a49-1c54-4d7f-850b-e3009c5ac443 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-4293-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [4293] Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4293] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4293] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4293] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=250 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 250, Current: 250, Stored: 250 Feb 17 18:40:18 [4293] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=250 for localhost Feb 17 18:40:18 [4293] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-4293-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-4293-11) state:2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-4293-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-4293-11-header Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource sysinfo_node-1.test.domain.local are unavailable, unclean or shutting down (node-1.test.domain.local: 0, 1000000) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for sysinfo_node-1.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: Resource sysinfo_node-1.test.domain.local cannot run anywhere Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-4293-11-header Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_vrouter:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_vrouter:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_vrouter:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_vrouter:2 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: Resource p_vrouter:2 cannot run anywhere Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_vrouter instances of a possible 3 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: clone_p_haproxy: Rolling back scores from vip__public Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_haproxy:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_haproxy:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_haproxy:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_haproxy:2 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: Resource p_haproxy:2 cannot run anywhere Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_haproxy instances of a possible 3 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__management Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter_pub Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource vip__public are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for vip__public Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: Resource vip__public cannot run anywhere Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_conntrackd:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_conntrackd:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: p_conntrackd:2: Rolling back scores from vip__vrouter_pub Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_conntrackd:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_conntrackd:2 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: Resource p_conntrackd:2 cannot run anywhere Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_conntrackd instances of a possible 3 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: master_p_conntrackd: Breaking dependency loop at master_p_conntrackd Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:1 master score: 1000000 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_conntrackd:1 (Master node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:2 master score: 0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:0 master score: -1000000 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: master_color: master_p_conntrackd: Promoted 1 instances of a possible 1 to master Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_mysql:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_mysql:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_mysql:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_mysql:2 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: Resource p_mysql:2 cannot run anywhere Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_mysql instances of a possible 3 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_rabbitmq-server:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_rabbitmq-server:2 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: Resource p_rabbitmq-server:2 cannot run anywhere Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_rabbitmq-server instances of a possible 3 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:0 master score: 1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_rabbitmq-server:0 (Slave node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:1 master score: 1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:2 master score: 0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: master_color: master_p_rabbitmq-server: Promoted 1 instances of a possible 1 to master Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:0 with p_vrouter:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_dns:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:1 with p_vrouter:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_dns:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:2 with p_vrouter:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_dns:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_dns:2 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: Resource p_dns:2 cannot run anywhere Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_dns instances of a possible 3 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_heat-engine:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_heat-engine:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_heat-engine:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_heat-engine:2 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: Resource p_heat-engine:2 cannot run anywhere Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_heat-engine instances of a possible 3 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-plugin-openvswitch-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-plugin-openvswitch-agent:2 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-plugin-openvswitch-agent:2 cannot run anywhere Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-plugin-openvswitch-agent instances of a possible 3 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-l3-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-l3-agent:2 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-l3-agent:2 cannot run anywhere Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-l3-agent instances of a possible 3 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-dhcp-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-dhcp-agent:2 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-dhcp-agent:2 cannot run anywhere Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-dhcp-agent instances of a possible 3 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-metadata-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-metadata-agent:2 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-metadata-agent:2 cannot run anywhere Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-metadata-agent instances of a possible 3 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to sysinfo_node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to sysinfo_node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to ping_vip__public:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to ping_vip__public:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource ping_vip__public:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for ping_vip__public:2 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: Resource ping_vip__public:2 cannot run anywhere Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_ping_vip__public instances of a possible 3 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_ntp:0 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_ntp:1 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_ntp:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_ntp:2 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: native_color: Resource p_ntp:2 cannot run anywhere Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_ntp instances of a possible 3 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_conntrackd Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:0 on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:1 on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_rabbitmq-server Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:1 on node-3.test.domain.local Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:1 (Master node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:0 (node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:1 (node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_rabbitmq-server:0 (Slave -> Master node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:1 (Slave node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. (null)) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. (null)) Feb 17 18:40:18 [4305] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. ping_vip__public) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:1 (Master node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:1 (Slave node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: debug: cib_file_signoff: Signing out of the CIB Service Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=4305 id=82e637a0-c24f-42ae-8fb3-d20ac11a26d0 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-4305-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [4305] Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: cib_file_signoff: Wrote CIB to /var/lib/pacemaker/cib/shadow.4275 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4305] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4305] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4305] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11686 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11686, Current: 11686, Stored: 11686 Feb 17 18:40:18 [4305] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11686 for localhost Feb 17 18:40:18 [4305] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [4275] node-3.test.domain.local crm_simulate: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-4305-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-4305-11) state:2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-4305-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-4305-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-4305-11-header Feb 17 18:40:18 [4321] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=4321 id=c47b1269-a88d-48ab-829c-8a67126abdb4 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-4321-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [4321] Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4321] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4321] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4321] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11686 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11686, Current: 11686, Stored: 11686 Feb 17 18:40:18 [4321] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11686 for localhost Feb 17 18:40:18 [4321] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-4321-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-4321-11) state:2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-4321-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-4321-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-4321-11-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=4318 id=873d7e8f-6147-4a04-bd78-d23b0272656a Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-4318-14) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [4318] Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/114) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [4337] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-4318-14) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-4318-14) state:2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4318-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4318-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4318-14-header Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4318-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.566 2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.567 (null) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=567 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-root_free']: @value=11681 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/813, version=0.155.567) Feb 17 18:40:18 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.567 to 0.155.566 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=4337 id=11c6fec1-f16e-4984-97e3-5c4fa10976ae Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-4337-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [4337] Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4318-14-header Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4318-14-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4337] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4337] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4337] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9069 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9069, Current: 9070, Stored: 9070 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9069 Feb 17 18:40:18 [4337] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9069 for localhost Feb 17 18:40:18 [4337] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9069) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: validate_with_relaxng: Creating RNG parser context Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/114, version=0.155.567) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 116: var_log_free=9069 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/116) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 114 for root_free=11686 passed Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-4337-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-4337-11) state:2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-4337-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-4337-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-4337-11-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e080 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.567 2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.568 (null) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=568 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9069 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/116, version=0.155.568) Feb 17 18:40:18 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.568 to 0.155.567 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 116 for var_log_free=9069 passed Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e080 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/815, version=0.155.568) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [4352] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df20 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.568 2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.569 (null) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=569 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9067 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/817, version=0.155.569) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:40:18 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.569 to 0.155.568 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=4352 id=87eb44fe-ef72-4e2d-8e63-e0a818f413f0 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-4352-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [4352] Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/118) Feb 17 18:40:18 [4352] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4352] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4352] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18422 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18422, Current: 18422, Stored: 18422 Feb 17 18:40:18 [4352] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18422 for localhost Feb 17 18:40:18 [4352] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-4352-11) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-4352-11) state:2 Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-4352-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-4352-11-header Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-4352-11-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df20 Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(4029) = 0: Resource temporarily unavailable (11) Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(4049) = 0: Resource temporarily unavailable (11) Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:4185 - exited with rc=0 Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:4185:stderr [ -- empty -- ] Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:4185:stdout [ arch: x86_64 ] Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:4185:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:4185:stdout [ free_swap: 3100 ] Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:4185:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:4185:stdout [ cpu_speed: 6999.99 ] Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:4185:stdout [ cpu_cores: 2 ] Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:4185:stdout [ cpu_load: 0.68, ] Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:4185:stdout [ ram_total: 3050 ] Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:4185:stdout [ ram_free: 250 ] Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:4185:stdout [ root_free: 11686 ] Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:4185:stdout [ root_free: 11686 ] Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:4185:stdout [ var_log_free: 9069 ] Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:4185:stdout [ var_lib_mysql_free: 18422 ] Feb 17 18:40:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:4185 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: cib_file_signon: crm_simulate: Opened connection to local file '/var/lib/pacemaker/cib/shadow.4318' Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: cib_file_perform_op_delegate: cib_query on (null) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: unpack_config: Stop all active resources: false Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: unpack_config: Default stickiness: 0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/118, version=0.155.569) Feb 17 18:40:18 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 118 for var_log_free=9069 passed Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_mysql_monitor_60000 on node-2.test.domain.local returned 'unknown error' (1) instead of the expected value: 'ok' (0) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:0 on node-3.test.domain.local: unknown error (1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-3.test.domain.local to ping_vip__public:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-3.test.domain.local to p_ntp:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: apply_system_health: Applying automated node health strategy: migrate-on-red Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-1.test.domain.local has an combined system health of 0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-2.test.domain.local has an combined system health of 0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-3.test.domain.local has an combined system health of 0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-1.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_vrouter [p_vrouter] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_print: vip__management (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter_pub (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_print: vip__public (ocf::fuel:ns_IPaddr2): Stopped Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_conntrackd [p_conntrackd] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: short_print: Masters: [ node-3.test.domain.local ] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: short_print: Slaves: [ node-2.test.domain.local ] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_haproxy [p_haproxy] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_mysql [p_mysql] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: short_print: Slaves: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_dns [p_dns] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_heat-engine [p_heat-engine] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-plugin-openvswitch-agent [p_neutron-plugin-openvswitch-agent] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-l3-agent [p_neutron-l3-agent] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-dhcp-agent [p_neutron-dhcp-agent] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-metadata-agent [p_neutron-metadata-agent] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-3.test.domain.local (ocf::pacemaker:SysInfo): Started node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-2.test.domain.local (ocf::pacemaker:SysInfo): Started node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_ping_vip__public [ping_vip__public] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_ntp [p_ntp] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__management: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 38 times on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 38 times on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 38 times on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_rabbitmq-server:0: preferring current location (node=node-2.test.domain.local, weight=100) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_heat-engine:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-plugin-openvswitch-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-l3-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:0 has failed 4 times on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:1 has failed 4 times on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-dhcp-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-metadata-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource ping_vip__public:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_ntp:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter_pub: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 4 times on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999996 more times on node-3.test.domain.local before being forced off Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 4 times on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999996 more times on node-3.test.domain.local before being forced off Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 4 times on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999996 more times on node-3.test.domain.local before being forced off Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_rabbitmq-server:1: preferring current location (node=node-3.test.domain.local, weight=100) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_heat-engine:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-plugin-openvswitch-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-l3-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-dhcp-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-metadata-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource ping_vip__public:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_ntp:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource sysinfo_node-1.test.domain.local are unavailable, unclean or shutting down (node-1.test.domain.local: 0, 1000000) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for sysinfo_node-1.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: Resource sysinfo_node-1.test.domain.local cannot run anywhere Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_vrouter:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_vrouter:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_vrouter:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_vrouter:2 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: Resource p_vrouter:2 cannot run anywhere Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_vrouter instances of a possible 3 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: clone_p_haproxy: Rolling back scores from vip__public Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_haproxy:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_haproxy:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_haproxy:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_haproxy:2 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: Resource p_haproxy:2 cannot run anywhere Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_haproxy instances of a possible 3 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__management Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter_pub Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource vip__public are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for vip__public Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: Resource vip__public cannot run anywhere Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_conntrackd:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_conntrackd:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: p_conntrackd:2: Rolling back scores from vip__vrouter_pub Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_conntrackd:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_conntrackd:2 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: Resource p_conntrackd:2 cannot run anywhere Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_conntrackd instances of a possible 3 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: master_p_conntrackd: Breaking dependency loop at master_p_conntrackd Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:1 master score: 1000000 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_conntrackd:1 (Master node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:2 master score: 0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:0 master score: -1000000 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: master_color: master_p_conntrackd: Promoted 1 instances of a possible 1 to master Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_mysql:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_mysql:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_mysql:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_mysql:2 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: Resource p_mysql:2 cannot run anywhere Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_mysql instances of a possible 3 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_rabbitmq-server:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_rabbitmq-server:2 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: Resource p_rabbitmq-server:2 cannot run anywhere Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_rabbitmq-server instances of a possible 3 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:0 master score: 1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_rabbitmq-server:0 (Slave node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:1 master score: 1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:2 master score: 0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: master_color: master_p_rabbitmq-server: Promoted 1 instances of a possible 1 to master Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:0 with p_vrouter:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_dns:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:1 with p_vrouter:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_dns:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:2 with p_vrouter:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_dns:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_dns:2 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: Resource p_dns:2 cannot run anywhere Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_dns instances of a possible 3 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_heat-engine:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_heat-engine:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_heat-engine:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_heat-engine:2 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: Resource p_heat-engine:2 cannot run anywhere Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_heat-engine instances of a possible 3 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-plugin-openvswitch-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-plugin-openvswitch-agent:2 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-plugin-openvswitch-agent:2 cannot run anywhere Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-plugin-openvswitch-agent instances of a possible 3 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-l3-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-l3-agent:2 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-l3-agent:2 cannot run anywhere Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-l3-agent instances of a possible 3 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-dhcp-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-dhcp-agent:2 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-dhcp-agent:2 cannot run anywhere Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-dhcp-agent instances of a possible 3 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-metadata-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-metadata-agent:2 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-metadata-agent:2 cannot run anywhere Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-metadata-agent instances of a possible 3 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to sysinfo_node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to sysinfo_node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to ping_vip__public:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to ping_vip__public:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource ping_vip__public:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for ping_vip__public:2 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: Resource ping_vip__public:2 cannot run anywhere Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_ping_vip__public instances of a possible 3 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_ntp:0 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_ntp:1 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_ntp:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_ntp:2 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: native_color: Resource p_ntp:2 cannot run anywhere Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_ntp instances of a possible 3 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_conntrackd Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:0 on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:1 on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_rabbitmq-server Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:1 on node-3.test.domain.local Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:1 (Master node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:0 (node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:1 (node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_rabbitmq-server:0 (Slave -> Master node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:1 (Slave node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. (null)) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. (null)) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. ping_vip__public) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:1 (Master node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:1 (Slave node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:0 (Started node-2.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:1 (Started node-3.test.domain.local) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: debug: cib_file_signoff: Signing out of the CIB Service Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: cib_file_signoff: Wrote CIB to /var/lib/pacemaker/cib/shadow.4318 Feb 17 18:40:18 [4318] node-3.test.domain.local crm_simulate: info: crm_xml_cleanup: Cleaning up memory from libxml2 ocf-mysql-wss: 2016/02/17_18:40:18 INFO: Possible masters: node-2.test.domain.local ocf-mysql-wss: 2016/02/17_18:40:18 INFO: Choosed master: node-2.test.domain.local Feb 17 18:40:18 [4398] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:40:18 [4398] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:40:18 [4398] node-3.test.domain.local crm_node: debug: main: Attempting to process -q command for cluster type: corosync Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-4398-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [4398] Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [4398] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [4398] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [4398] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7f761349cf10 Feb 17 18:40:18 [4398] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [4398] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-request-6268-4398-26-header Feb 17 18:40:18 [4398] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-response-6268-4398-26-header Feb 17 18:40:18 [4398] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-event-6268-4398-26-header Feb 17 18:40:18 [4398] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-4398-26) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-4398-26) state:2 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7f761349cf10 Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-6268-4398-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-6268-4398-26-header Feb 17 18:40:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-6268-4398-26-header Feb 17 18:40:18 [4407] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:40:18 [4407] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:40:18 [4407] node-3.test.domain.local crm_node: debug: main: Attempting to process -l command for cluster type: corosync Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=4407 id=be9a57c4-8fcc-4637-b3e1-161d5a8f8952 Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-4407-12) Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [4407] Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4407] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4407] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4407] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:18 [4407] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-4407-12) Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-4407-12) state:2 Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-4407-12-header Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-4407-12-header Feb 17 18:40:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-4407-12-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=4412 id=a7ae22ec-93da-4329-a29d-b0d2261ce478 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-4412-14) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [4412] Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: unpack_config: Stop all active resources: false Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: unpack_config: Default stickiness: 0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: determine_op_status: p_mysql_monitor_60000 on node-2.test.domain.local returned 'unknown error' (1) instead of the expected value: 'ok' (0) Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:0 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:0 on node-3.test.domain.local: unknown error (1) Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:1 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:1 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-3.test.domain.local to ping_vip__public:1 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-3.test.domain.local to p_ntp:1 Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-4412-14) Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-4412-14) state:2 Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4412-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4412-14-header Feb 17 18:40:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4412-14-header Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-4412-14-header Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-4412-14-header Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-4412-14-header Feb 17 18:40:18 [4412] node-3.test.domain.local crm_resource: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df20 Feb 17 18:40:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:40:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:40:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df20 Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.569 2 Feb 17 18:40:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.570 (null) Feb 17 18:40:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=570 Feb 17 18:40:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/819, version=0.155.570) Feb 17 18:40:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:19 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit-start-time'] does not exist Feb 17 18:40:19 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for rabbit-start-time=(null) passed Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:19 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.570 to 0.155.569 Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_IPaddr2: 2016/02/17_18:40:19 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.671ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.700ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.537ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.531ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:40:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:40:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:4029 - exited with rc=0 Feb 17 18:40:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:4029:stderr [ -- empty -- ] Feb 17 18:40:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:4029:stdout [ -- empty -- ] Feb 17 18:40:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:4029 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(4049) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:40:19 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.785ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.537ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.528ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:40:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:4049 - exited with rc=0 Feb 17 18:40:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:4049:stderr [ -- empty -- ] Feb 17 18:40:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:4049:stdout [ -- empty -- ] Feb 17 18:40:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:4049 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:40:20 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff Feb 17 18:40:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:4554 - exited with rc=8 Feb 17 18:40:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:4554:stderr [ -- empty -- ] Feb 17 18:40:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:4554:stdout [ vrouter ] Feb 17 18:40:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:4554:stdout [ mgmt-conntrd ] Feb 17 18:40:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:4554 exit-code:8 exec-time:0ms queue-time:0ms Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:21 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:21 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:21 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:21 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:21 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:21 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.570 2 Feb 17 18:40:21 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.571 (null) Feb 17 18:40:21 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=571 Feb 17 18:40:21 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_rabbitmq-server']/lrm_rsc_op[@id='p_rabbitmq-server_last_0']: @operation_key=p_rabbitmq-server_promote_0, @operation=promote, @transition-key=90:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;90:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=285, @last-run=1455734408, @last-rc-change=1455734408, @exec-time=13767 Feb 17 18:40:21 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/376, version=0.155.571) Feb 17 18:40:21 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.571 to 0.155.570 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:21 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:21 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:21 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:21 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:21 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:21 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:21 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:21 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:21 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=237:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_rabbitmq-server_notify_0 Feb 17 18:40:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=137, reply=1, notify=0, exit=4202464 Feb 17 18:40:21 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_rabbitmq-server action:notify call_id:137 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:21 [4669] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:40:21 [4669] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-4669-26) Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [4669] Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761349cf10, cpd=0x7f761349d7f4 Feb 17 18:40:21 [4669] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [4669] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [4669] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f761349cf10 Feb 17 18:40:21 [4669] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:21 [4669] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-6268-4669-26-header Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-4669-26) Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-4669-26) state:2 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-4669-26-header Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-4669-26-header Feb 17 18:40:21 [4669] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-6268-4669-26-header Feb 17 18:40:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-4669-26-header Feb 17 18:40:21 [4669] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-6268-4669-26-header Feb 17 18:40:22 [4669] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-4669-26) Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [4669] Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:22 [4669] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:22 [4669] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:22 [4669] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:22 [4669] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:22 [4669] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-4669-26-header Feb 17 18:40:22 [4669] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-4669-26-header Feb 17 18:40:22 [4669] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-4669-26-header Feb 17 18:40:22 [4669] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:40:22 [4669] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:40:22 [4669] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-4669-26) Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-4669-26) state:2 Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-4669-26-header Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-4669-26-header Feb 17 18:40:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-4669-26-header lrmd: 2016/02/17_18:40:22 INFO: p_rabbitmq-server: notify: post-promote begin. lrmd: 2016/02/17_18:40:22 INFO: p_rabbitmq-server: my_host(): hostlist is: node-2.test.domain.local lrmd: 2016/02/17_18:40:22 DEBUG: p_rabbitmq-server: my_host(): comparing 'messaging-node-3' with 'messaging-node-2' lrmd: 2016/02/17_18:40:22 DEBUG: p_rabbitmq-server: jjj_join: node='node-2.test.domain.local' rc='10' lrmd: 2016/02/17_18:40:22 INFO: p_rabbitmq-server: jjj_join: Joining to cluster by node 'node-2.test.domain.local' lrmd: 2016/02/17_18:40:22 INFO: p_rabbitmq-server: join_to_cluster(): start. lrmd: 2016/02/17_18:40:22 INFO: p_rabbitmq-server: join_to_cluster(): Joining to cluster by node ''. lrmd: 2016/02/17_18:40:22 INFO: get_status(): app rabbit was not found in command output: [{xmerl,"XML parser","1.3.8"}, {inets,"INETS CXC 138 49","6.0.1"}, {sasl,"SASL CXC 138 11","2.6"}, {stdlib,"ERTS CXC 138 10","2.6"}, {kernel,"ERTS CXC 138 10","4.1"}] lrmd: 2016/02/17_18:40:22 INFO: p_rabbitmq-server: join_to_cluster(): Execute join_cluster with timeout: 60 lrmd: 2016/02/17_18:40:22 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl join_cluster rabbit@messaging-node-2 Feb 17 18:40:23 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.006000 (18 ticks in 30s) Feb 17 18:40:23 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.980000 (full: 0.98 0.69 0.37 1/487 5067) Feb 17 18:40:23 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 lrmd: 2016/02/17_18:40:23 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl join_cluster rabbit@messaging-node-2 Feb 17 18:40:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:40:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='pingd'] does not exist Feb 17 18:40:25 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 121: pingd=1000 Feb 17 18:40:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/121) Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.571 2 Feb 17 18:40:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.572 (null) Feb 17 18:40:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=572 Feb 17 18:40:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Feb 17 18:40:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/821, version=0.155.572) Feb 17 18:40:25 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.572 to 0.155.571 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.572 2 Feb 17 18:40:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.573 (null) Feb 17 18:40:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=573 Feb 17 18:40:25 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:40:25 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/121, version=0.155.573) Feb 17 18:40:25 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.573 to 0.155.572 Feb 17 18:40:25 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 121 for pingd=1000 passed Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header lrmd: 2016/02/17_18:40:25 INFO: p_rabbitmq-server: try_to_start_rmq_app(): begin. lrmd: 2016/02/17_18:40:25 INFO: p_rabbitmq-server: try_to_start_rmq_app(): Execute start_app with timeout: 60 lrmd: 2016/02/17_18:40:25 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl start_app >>/var/log/rabbitmq/startup_log 2>&1 lrmd: 2016/02/17_18:40:26 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl start_app >>/var/log/rabbitmq/startup_log 2>&1 lrmd: 2016/02/17_18:40:26 INFO: p_rabbitmq-server: try_to_start_rmq_app(): start_app was successful. lrmd: 2016/02/17_18:40:26 INFO: p_rabbitmq-server: try_to_start_rmq_app(): waiting for start to finish with timeout: 60 lrmd: 2016/02/17_18:40:26 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl wait /var/run/rabbitmq/p_pid Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:26 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.573 2 Feb 17 18:40:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.574 (null) Feb 17 18:40:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=574 Feb 17 18:40:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_mysql']/lrm_rsc_op[@id='p_mysql_last_0']: @operation_key=p_mysql_start_0, @operation=start, @transition-key=80:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;80:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=283, @last-run=1455734407, @last-rc-change=1455734407, @exec-time=18882 Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:26 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/377, version=0.155.574) Feb 17 18:40:26 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.574 to 0.155.573 Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_IPaddr2: 2016/02/17_18:40:27 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.801ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.833ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.540ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.534ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:40:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:40:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(4650) = 0: Interrupted system call (4) Feb 17 18:40:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:5140 - exited with rc=0 Feb 17 18:40:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:5140:stderr [ -- empty -- ] Feb 17 18:40:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:5140:stdout [ -- empty -- ] Feb 17 18:40:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:5140 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(5159) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:40:27 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.970ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.528ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:40:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(4650) = 0: Resource temporarily unavailable (11) Feb 17 18:40:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:5159 - exited with rc=0 Feb 17 18:40:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:5159:stderr [ -- empty -- ] Feb 17 18:40:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:5159:stdout [ -- empty -- ] Feb 17 18:40:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:5159 exit-code:0 exec-time:0ms queue-time:0ms lrmd: 2016/02/17_18:40:27 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl wait /var/run/rabbitmq/p_pid lrmd: 2016/02/17_18:40:27 INFO: p_rabbitmq-server: try_to_start_rmq_app(): start plugins. Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_ntp: 2016/02/17_18:40:28 INFO: ntp daemon running Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(4650) = 0: Resource temporarily unavailable (11) Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:5901 - exited with rc=0 Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:5901:stderr [ -- empty -- ] Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:5901:stdout [ -- empty -- ] Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:5901 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(5913) = 0: Resource temporarily unavailable (11) ocf-ns_vrouter: 2016/02/17_18:40:28 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:40:28 INFO: net.ipv4.ip_forward = 1 Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(4650) = 0: Resource temporarily unavailable (11) Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:5913 - exited with rc=0 Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:5913:stderr [ -- empty -- ] Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:5913:stdout [ -- empty -- ] Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:5913 exit-code:0 exec-time:0ms queue-time:0ms lrmd: 2016/02/17_18:40:28 INFO: p_rabbitmq-server: try_to_start_rmq_app(): Starting plugins: [rabbitmq_management,rabbitmq_management_agent,rabbitmq_web_dispatch, webmachine,mochiweb,amqp_client] lrmd: 2016/02/17_18:40:28 INFO: p_rabbitmq-server: join_to_cluster(): Rabbit app started successfully. Updating start time attribute with 1455734428 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=5971 id=db18ab78-3fe6-4f5d-a079-530e35ca4702 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-5971-14) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [5971] Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=5971 id=f12a0d03-172c-4260-9bb1-98c0c58d80f2 Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-5971-11) Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [5971] Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit-start-time=1455734428 Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1455734428, Current: (null), Stored: (null) Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of rabbit-start-time is 1455734428 Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: rabbit-start-time (1455734428) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit-start-time'] does not exist Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 123: rabbit-start-time=1455734428 Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit-start-time=1455734428 for node-3.test.domain.local Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: info: main: Update rabbit-start-time=1455734428 sent via attrd Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-5971-14-header Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-5971-14-header Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-5971-14-header Feb 17 18:40:28 [5971] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/123) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-5971-14) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-5971-14) state:2 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-5971-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-5971-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-5971-14-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-5971-11) Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-5971-11) state:2 Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-5971-11-header Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-5971-11-header Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-5971-11-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] ocf-mysql-wss: 2016/02/17_18:40:28 INFO: Waiting for master. 290 seconds left Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [5980] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:40:28 [5980] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:40:28 [5980] node-3.test.domain.local crm_node: debug: main: Attempting to process -p command for cluster type: corosync lrmd: 2016/02/17_18:40:28 INFO: p_rabbitmq-server: join_to_cluster(): Joined to cluster succesfully. Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.574 2 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.575 (null) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=575 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/123, version=0.155.575) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:28 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.575 to 0.155.574 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 123 for rabbit-start-time=1455734428 passed Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=5980 id=9841c15a-06d7-4cb2-860e-33d9b14a82af Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-5980-12) Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [5980] Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [5980] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [5980] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [5980] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/823, version=0.155.575) Feb 17 18:40:28 [5980] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-5980-12) Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-5980-12) state:2 Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-5980-12-header Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-5980-12-header Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-5980-12-header lrmd: 2016/02/17_18:40:28 INFO: p_rabbitmq-server: join_to_cluster(): end. lrmd: 2016/02/17_18:40:28 INFO: p_rabbitmq-server: notify: post-promote end. Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_notify_0:4650 - exited with rc=0 Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:4650:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:4650:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:4650:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:4650:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:4650:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:4650:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_notify_0:4650:stdout [ Clustering node 'rabbit@messaging-node-3' with 'rabbit@messaging-node-2' ... ] Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_notify_0:4650:stdout [ Waiting for 'rabbit@messaging-node-3' ... ] Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_notify_0:4650:stdout [ pid is 12355 ... ] Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_notify_0:4650:stdout [ false ] Feb 17 18:40:28 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_rabbitmq-server action:notify call_id:137 pid:4650 exit-code:0 exec-time:6607ms queue-time:0ms Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_rabbitmq-server after notify op complete (interval=0) Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_rabbitmq-server_notify_0 from 237:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455734428-20 Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_rabbitmq-server_notify_0: ok (node=node-3.test.domain.local, call=137, rc=0, cib-update=0, confirmed=true) Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_rabbitmq-server_notify_0:137 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n ] Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=5992 id=f39c5d7d-a429-4295-9813-23b0c66492db Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-5992-14) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [5992] Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-2.test.domain.local to 2 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='2']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: info: main: Read gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:20770 Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-5992-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-5992-14) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-5992-14) state:2 Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-5992-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-5992-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-5992-14-header Feb 17 18:40:28 [5992] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-5992-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-5992-14-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] ocf-mysql-wss: 2016/02/17_18:40:28 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:20770 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:28 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header ocf-mysql-wss: 2016/02/17_18:40:28 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:20770 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=6013 id=ec4bf9a8-c92b-4b05-af2d-939d0e931f9c Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-6013-14) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [6013] Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[7]) Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: info: main: Read gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-6013-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-6013-14) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-6013-14) state:2 Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-6013-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-6013-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-6013-14-header Feb 17 18:40:28 [6013] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-6013-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-6013-14-header ocf-mysql-wss: 2016/02/17_18:40:28 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1 ocf-mysql-wss: 2016/02/17_18:40:28 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dd10 for uid=0 gid=0 pid=6033 id=a48be428-a261-4102-981b-8256bcf9cf4b Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-6033-14) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [6033] Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-6033-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-6033-14) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-6033-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-6033-14) state:2 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-6033-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-6033-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-6033-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-6033-14-header Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: validate_with_relaxng: Creating RNG parser context Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: cib_file_signon: crm_simulate: Opened connection to local file '/var/lib/pacemaker/cib/shadow.6033' Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: cib_file_perform_op_delegate: cib_query on (null) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: unpack_config: Stop all active resources: false Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: unpack_config: Default stickiness: 0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_mysql_monitor_60000 on node-2.test.domain.local returned 'unknown error' (1) instead of the expected value: 'ok' (0) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:1 on node-3.test.domain.local: unknown error (1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-3.test.domain.local to ping_vip__public:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-3.test.domain.local to p_ntp:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: apply_system_health: Applying automated node health strategy: migrate-on-red Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-1.test.domain.local has an combined system health of 0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-2.test.domain.local has an combined system health of 0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-3.test.domain.local has an combined system health of 0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-1.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_vrouter [p_vrouter] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_print: vip__management (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter_pub (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_print: vip__public (ocf::fuel:ns_IPaddr2): Stopped Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_conntrackd [p_conntrackd] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Masters: [ node-3.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Slaves: [ node-2.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_haproxy [p_haproxy] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_mysql [p_mysql] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-3.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Masters: [ node-2.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Slaves: [ node-3.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_dns [p_dns] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_heat-engine [p_heat-engine] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-plugin-openvswitch-agent [p_neutron-plugin-openvswitch-agent] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-l3-agent [p_neutron-l3-agent] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-dhcp-agent [p_neutron-dhcp-agent] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-metadata-agent [p_neutron-metadata-agent] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-3.test.domain.local (ocf::pacemaker:SysInfo): Started node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-2.test.domain.local (ocf::pacemaker:SysInfo): Started node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_ping_vip__public [ping_vip__public] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_ntp [p_ntp] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__management: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_mysql:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 38 times on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:1 has failed 38 times on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 38 times on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_rabbitmq-server:0: preferring current location (node=node-2.test.domain.local, weight=100) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_heat-engine:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-plugin-openvswitch-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-l3-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:0 has failed 4 times on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:1 has failed 4 times on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-dhcp-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-metadata-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource ping_vip__public:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_ntp:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter_pub: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 4 times on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999996 more times on node-3.test.domain.local before being forced off Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:1 has failed 4 times on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999996 more times on node-3.test.domain.local before being forced off Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 4 times on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999996 more times on node-3.test.domain.local before being forced off Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_rabbitmq-server:1: preferring current location (node=node-3.test.domain.local, weight=100) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_heat-engine:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-plugin-openvswitch-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-l3-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-dhcp-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-metadata-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource ping_vip__public:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_ntp:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource sysinfo_node-1.test.domain.local are unavailable, unclean or shutting down (node-1.test.domain.local: 0, 1000000) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for sysinfo_node-1.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_color: Resource sysinfo_node-1.test.domain.local cannot run anywhere Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_vrouter:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_vrouter:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_vrouter:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_vrouter:2 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_color: Resource p_vrouter:2 cannot run anywhere Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_vrouter instances of a possible 3 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_haproxy:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_haproxy:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_haproxy:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_haproxy:2 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_color: Resource p_haproxy:2 cannot run anywhere Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_haproxy instances of a possible 3 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__management Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter_pub Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__public Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_conntrackd:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_conntrackd:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_color: p_conntrackd:2: Rolling back scores from vip__vrouter_pub Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_conntrackd:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_conntrackd:2 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_color: Resource p_conntrackd:2 cannot run anywhere Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_conntrackd instances of a possible 3 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: master_p_conntrackd: Breaking dependency loop at master_p_conntrackd Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:1 master score: 1000000 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_conntrackd:1 (Master node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:2 master score: 0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:0 master score: -1000000 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: master_color: master_p_conntrackd: Promoted 1 instances of a possible 1 to master Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_mysql:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_mysql:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_mysql:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_mysql:2 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_color: Resource p_mysql:2 cannot run anywhere Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_mysql instances of a possible 3 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_rabbitmq-server:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_rabbitmq-server:2 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_color: Resource p_rabbitmq-server:2 cannot run anywhere Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_rabbitmq-server instances of a possible 3 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:0 master score: 1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_rabbitmq-server:0 (Master node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:1 master score: 1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:2 master score: 0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: master_color: master_p_rabbitmq-server: Promoted 1 instances of a possible 1 to master Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:0 with p_vrouter:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_dns:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:1 with p_vrouter:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_dns:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:2 with p_vrouter:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_dns:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_dns:2 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_color: Resource p_dns:2 cannot run anywhere Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_dns instances of a possible 3 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_heat-engine:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_heat-engine:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_heat-engine:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_heat-engine:2 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_color: Resource p_heat-engine:2 cannot run anywhere Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_heat-engine instances of a possible 3 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-plugin-openvswitch-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-plugin-openvswitch-agent:2 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-plugin-openvswitch-agent:2 cannot run anywhere Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-plugin-openvswitch-agent instances of a possible 3 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-l3-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-l3-agent:2 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-l3-agent:2 cannot run anywhere Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-l3-agent instances of a possible 3 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-dhcp-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-dhcp-agent:2 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-dhcp-agent:2 cannot run anywhere Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-dhcp-agent instances of a possible 3 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-metadata-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-metadata-agent:2 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-metadata-agent:2 cannot run anywhere Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-metadata-agent instances of a possible 3 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to sysinfo_node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to sysinfo_node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to ping_vip__public:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to ping_vip__public:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource ping_vip__public:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for ping_vip__public:2 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_color: Resource ping_vip__public:2 cannot run anywhere Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_ping_vip__public instances of a possible 3 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_ntp:0 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_ntp:1 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_ntp:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_ntp:2 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: native_color: Resource p_ntp:2 cannot run anywhere Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_ntp instances of a possible 3 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (5s) for vip__public on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_conntrackd Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:0 on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:1 on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_rabbitmq-server Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:1 on node-3.test.domain.local Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: notice: LogActions: Start vip__public (node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:1 (Master node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:1 (node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:0 (Master node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:1 (Slave node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. p_mysql) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. p_mysql) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. ping_vip__public) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:1 (Master node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:0 (Master node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:1 (Slave node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: debug: cib_file_signoff: Signing out of the CIB Service Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: cib_file_signoff: Wrote CIB to /var/lib/pacemaker/cib/shadow.6033 Feb 17 18:40:28 [6033] node-3.test.domain.local crm_simulate: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dd10 for uid=0 gid=0 pid=6039 id=30cc953e-2781-4032-ae02-8c616b09af1d Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-6039-14) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [6039] Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-6039-14) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-6039-14) state:2 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-6039-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-6039-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-6039-14-header Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-6039-14-header Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-6039-14-header Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-6039-14-header Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: validate_with_relaxng: Creating RNG parser context Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: cib_file_signon: crm_simulate: Opened connection to local file '/var/lib/pacemaker/cib/shadow.6039' Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: cib_file_perform_op_delegate: cib_query on (null) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: unpack_config: Stop all active resources: false Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: unpack_config: Default stickiness: 0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_mysql_monitor_60000 on node-2.test.domain.local returned 'unknown error' (1) instead of the expected value: 'ok' (0) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:1 on node-3.test.domain.local: unknown error (1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-3.test.domain.local to ping_vip__public:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-3.test.domain.local to p_ntp:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: apply_system_health: Applying automated node health strategy: migrate-on-red Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-1.test.domain.local has an combined system health of 0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-2.test.domain.local has an combined system health of 0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-3.test.domain.local has an combined system health of 0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-1.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_vrouter [p_vrouter] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_print: vip__management (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter_pub (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter (ocf::fuel:ns_IPaddr2): Started node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_print: vip__public (ocf::fuel:ns_IPaddr2): Stopped Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_conntrackd [p_conntrackd] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_conntrackd:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Masters: [ node-3.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Slaves: [ node-2.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_haproxy [p_haproxy] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_mysql [p_mysql] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-3.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_rabbitmq-server:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Masters: [ node-2.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Slaves: [ node-3.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_dns [p_dns] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_heat-engine [p_heat-engine] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-plugin-openvswitch-agent [p_neutron-plugin-openvswitch-agent] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-plugin-openvswitch-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-l3-agent [p_neutron-l3-agent] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-l3-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-dhcp-agent [p_neutron-dhcp-agent] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-dhcp-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-metadata-agent [p_neutron-metadata-agent] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_neutron-metadata-agent:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-3.test.domain.local (ocf::pacemaker:SysInfo): Started node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-2.test.domain.local (ocf::pacemaker:SysInfo): Started node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_ping_vip__public [ping_vip__public] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource ping_vip__public:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_ntp [p_ntp] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:0 active on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_ntp:1 active on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__management: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_mysql:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 38 times on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:1 has failed 38 times on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 38 times on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_rabbitmq-server:0: preferring current location (node=node-2.test.domain.local, weight=100) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_heat-engine:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-plugin-openvswitch-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-l3-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:0 has failed 4 times on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:1 has failed 4 times on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-dhcp-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-metadata-agent:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource ping_vip__public:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_ntp:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter_pub: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_conntrackd:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 4 times on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999996 more times on node-3.test.domain.local before being forced off Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:1 has failed 4 times on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999996 more times on node-3.test.domain.local before being forced off Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 4 times on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999996 more times on node-3.test.domain.local before being forced off Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_rabbitmq-server:1: preferring current location (node=node-3.test.domain.local, weight=100) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_heat-engine:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-plugin-openvswitch-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-l3-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-dhcp-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_neutron-metadata-agent:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource ping_vip__public:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_ntp:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource sysinfo_node-1.test.domain.local are unavailable, unclean or shutting down (node-1.test.domain.local: 0, 1000000) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for sysinfo_node-1.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_color: Resource sysinfo_node-1.test.domain.local cannot run anywhere Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_vrouter:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_vrouter:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_vrouter:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_vrouter:2 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_color: Resource p_vrouter:2 cannot run anywhere Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_vrouter instances of a possible 3 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_haproxy:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_haproxy:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_haproxy:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_haproxy:2 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_color: Resource p_haproxy:2 cannot run anywhere Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_haproxy instances of a possible 3 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__management Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter_pub Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to vip__vrouter Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__public Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_conntrackd:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_conntrackd:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_color: p_conntrackd:2: Rolling back scores from vip__vrouter_pub Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_conntrackd:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_conntrackd:2 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_color: Resource p_conntrackd:2 cannot run anywhere Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_conntrackd instances of a possible 3 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: master_p_conntrackd: Breaking dependency loop at master_p_conntrackd Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:1 master score: 1000000 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_conntrackd:1 (Master node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:2 master score: 0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:0 master score: -1000000 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: master_color: master_p_conntrackd: Promoted 1 instances of a possible 1 to master Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_mysql:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_mysql:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_mysql:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_mysql:2 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_color: Resource p_mysql:2 cannot run anywhere Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_mysql instances of a possible 3 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_rabbitmq-server:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_rabbitmq-server:2 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_color: Resource p_rabbitmq-server:2 cannot run anywhere Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_rabbitmq-server instances of a possible 3 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:0 master score: 1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_rabbitmq-server:0 (Master node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:1 master score: 1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:2 master score: 0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: master_color: master_p_rabbitmq-server: Promoted 1 instances of a possible 1 to master Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:0 with p_vrouter:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_dns:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:1 with p_vrouter:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_dns:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:2 with p_vrouter:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_dns:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_dns:2 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_color: Resource p_dns:2 cannot run anywhere Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_dns instances of a possible 3 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_heat-engine:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_heat-engine:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_heat-engine:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_heat-engine:2 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_color: Resource p_heat-engine:2 cannot run anywhere Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_heat-engine instances of a possible 3 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-plugin-openvswitch-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-plugin-openvswitch-agent:2 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-plugin-openvswitch-agent:2 cannot run anywhere Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-plugin-openvswitch-agent instances of a possible 3 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-l3-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-l3-agent:2 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-l3-agent:2 cannot run anywhere Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-l3-agent instances of a possible 3 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-dhcp-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-dhcp-agent:2 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-dhcp-agent:2 cannot run anywhere Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-dhcp-agent instances of a possible 3 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-metadata-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-metadata-agent:2 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-metadata-agent:2 cannot run anywhere Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-metadata-agent instances of a possible 3 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to sysinfo_node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to sysinfo_node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to ping_vip__public:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to ping_vip__public:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource ping_vip__public:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for ping_vip__public:2 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_color: Resource ping_vip__public:2 cannot run anywhere Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_ping_vip__public instances of a possible 3 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_ntp:0 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_ntp:1 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_ntp:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_ntp:2 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: native_color: Resource p_ntp:2 cannot run anywhere Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_ntp instances of a possible 3 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (5s) for vip__public on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_conntrackd Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:0 on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:1 on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_rabbitmq-server Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-2.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:1 on node-3.test.domain.local Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: notice: LogActions: Start vip__public (node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:1 (Master node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:1 (node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:0 (Master node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:1 (Slave node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. p_mysql) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. p_mysql) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. p_rabbitmq-server) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. ping_vip__public) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:0 (Slave node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:1 (Master node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:0 (Master node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:1 (Slave node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:0 (Started node-2.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:1 (Started node-3.test.domain.local) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: debug: cib_file_signoff: Signing out of the CIB Service Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: cib_file_signoff: Wrote CIB to /var/lib/pacemaker/cib/shadow.6039 Feb 17 18:40:28 [6039] node-3.test.domain.local crm_simulate: info: crm_xml_cleanup: Cleaning up memory from libxml2 ocf-mysql-wss: 2016/02/17_18:40:28 INFO: Possible masters: node-2.test.domain.local ocf-mysql-wss: 2016/02/17_18:40:28 INFO: Choosed master: node-2.test.domain.local Feb 17 18:40:28 [6062] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:40:28 [6062] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:40:28 [6062] node-3.test.domain.local crm_node: debug: main: Attempting to process -q command for cluster type: corosync Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6062-26) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6062] Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6062] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6062] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6062] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7f761349cf10 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7f761349cf10 Feb 17 18:40:28 [6062] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:28 [6062] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-request-6268-6062-26-header Feb 17 18:40:28 [6062] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-response-6268-6062-26-header Feb 17 18:40:28 [6062] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-event-6268-6062-26-header Feb 17 18:40:28 [6062] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6062-26) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6062-26) state:2 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7f761349cf10 Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-6268-6062-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-6268-6062-26-header Feb 17 18:40:28 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-6268-6062-26-header Feb 17 18:40:28 [6071] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:40:28 [6071] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:40:28 [6071] node-3.test.domain.local crm_node: debug: main: Attempting to process -l command for cluster type: corosync Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=6071 id=d50b60e1-dfe1-42d2-8c64-91e88d11f4fd Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-6071-12) Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [6071] Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [6071] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [6071] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [6071] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:28 [6071] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-6071-12) Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-6071-12) state:2 Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-6071-12-header Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-6071-12-header Feb 17 18:40:28 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-6071-12-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dd10 for uid=0 gid=0 pid=6075 id=a4e1a1ef-ec6c-480f-a98b-4d58a6a28a50 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-6075-14) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [6075] Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: unpack_config: Stop all active resources: false Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: unpack_config: Default stickiness: 0 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: determine_op_status: p_mysql_monitor_60000 on node-2.test.domain.local returned 'unknown error' (1) instead of the expected value: 'ok' (0) Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:1 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:1 on node-3.test.domain.local: unknown error (1) Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:1 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:1 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-3.test.domain.local to ping_vip__public:1 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-3.test.domain.local to p_ntp:1 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-6075-14-header Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-6075-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-6075-14) Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-6075-14) state:2 Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-6075-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:28 [6075] node-3.test.domain.local crm_resource: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-6075-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-6075-14-header Feb 17 18:40:28 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-6075-14-header ocf-mysql-wss: 2016/02/17_18:40:28 INFO: My neighbour is Primary Component ocf-mysql-wss: 2016/02/17_18:40:28 INFO: Starting MySQL Feb 17 18:40:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 Feb 17 18:40:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-ns_dns: 2016/02/17_18:40:30 INFO: dnsmasq daemon running Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:6878 - exited with rc=0 Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:6878:stderr [ -- empty -- ] Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:6878:stdout [ -- empty -- ] Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:6878 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(6887) = 0: Resource temporarily unavailable (11) ocf-heat-engine: 2016/02/17_18:40:30 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:6887 - exited with rc=0 Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:6887:stderr [ -- empty -- ] Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:6887:stdout [ -- empty -- ] Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:6887 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:6922 - exited with rc=0 Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:6922:stderr [ -- empty -- ] Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:6922:stdout [ -- empty -- ] Feb 17 18:40:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:6922 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:40:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:40:33 [7261] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7261 id=4e639f35-c54e-4038-a684-d3c7c281332f Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7261-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7261] Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7261] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7261] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7261] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:40:33 [7261] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:40:33 [7261] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7261-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7261-11) state:2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7261-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7261-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7261-11-header Feb 17 18:40:33 [7268] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7268 id=5cbccd85-a7d9-4bf6-bc83-0b0e6c6ba2eb Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7268-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7268] Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7268] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7268] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7268] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:40:33 [7268] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:40:33 [7268] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7268-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7268-11) state:2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7268-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7268-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7268-11-header Feb 17 18:40:33 [7283] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7283 id=48455da6-fb6a-47d6-ba20-909445227aa7 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7283-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7283] Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7283] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7283] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7283] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7283] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3100 for localhost Feb 17 18:40:33 [7283] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3100 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3100, Current: 3100, Stored: 3100 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7283-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7283-11) state:2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7283-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7283-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7283-11-header Feb 17 18:40:33 [7290] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7290 id=078282dd-6e17-4dad-865b-1e5d7de1816f Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7290-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7290] Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7290] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7290] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7290] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:40:33 [7290] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:40:33 [7290] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7290-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7290-11) state:2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7290-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7290-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7290-11-header Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:40:33 [7291] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7291 id=ca662258-770e-493e-a70e-c59aa68d5679 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7291-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7291] Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7291] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7291] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7291] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:40:33 [7291] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:40:33 [7291] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7291-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7291-11) state:2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7291-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7291-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7291-11-header Feb 17 18:40:33 [7294] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7294 id=e1ed5e7f-a1a1-4b39-989b-6cde63c74d9a Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7294-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7294] Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7294] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7294] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7294] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:40:33 [7294] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:40:33 [7294] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7294-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7294-11) state:2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7294-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7294-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7294-11-header Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 Feb 17 18:40:33 [7303] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7303 id=0429503d-bbab-4ac9-a306-ce8aa5d5b1a7 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7303-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7303] Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7303] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7303] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7303] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.68, Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.68,, Current: 0.68,, Stored: 0.68, Feb 17 18:40:33 [7303] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.68, for localhost Feb 17 18:40:33 [7303] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7303-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7303-11) state:2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7303-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7303-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7303-11-header Feb 17 18:40:33 [7321] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7321 id=6ebf6d80-d792-4914-944e-eb205e19ea47 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7321-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7321] Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7321] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7321] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7321] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:40:33 [7321] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:40:33 [7321] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7321-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7321-11) state:2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7321-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7321-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7321-11-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(6983) = 0: Resource temporarily unavailable (11) Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7079) = 0: Resource temporarily unavailable (11) Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7253) = 0: Resource temporarily unavailable (11) Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:7292 - exited with rc=0 Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:7292:stderr [ -- empty -- ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:7292:stdout [ -- empty -- ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:7292 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7310) = 0: Resource temporarily unavailable (11) Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: c6df91ed3ee96805294342c0b7a55425 for 0.155.575 (0x1733cf0 0) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:33 [7347] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7347 id=13894c49-18c6-4170-982e-9a2088556354 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7347-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7347] Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7347] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7347] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7347] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=250 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 250, Current: 250, Stored: 250 Feb 17 18:40:33 [7347] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=250 for localhost Feb 17 18:40:33 [7347] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7347-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7347-11) state:2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7347-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7347-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7347-11-header ocf-neutron-dhcp-agent: 2016/02/17_18:40:33 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(6983) = 0: Resource temporarily unavailable (11) Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7079) = 0: Resource temporarily unavailable (11) Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7253) = 0: Resource temporarily unavailable (11) Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:7310 - exited with rc=0 Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:7310:stderr [ -- empty -- ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:7310:stdout [ -- empty -- ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:7310 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:33 [7374] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7374 id=fa786bd9-1648-4a69-a534-f25fdd2b4d31 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7374-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7374] Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7374] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7374] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7374] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11685 Feb 17 18:40:33 [7374] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11685 for localhost Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11685, Current: 11686, Stored: 11686 Feb 17 18:40:33 [7374] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of root_free is 11685 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: root_free (11685) Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/125) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 125: root_free=11685 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7374-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7374-11) state:2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7374-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7374-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7374-11-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349de50 Feb 17 18:40:33 [7386] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.575 2 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.576 (null) Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=576 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-root_free']: @value=11685 Feb 17 18:40:33 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.576 to 0.155.575 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7386 id=709ec72e-dd4e-4981-8282-15ae301f1dc4 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7386-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7386] Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349de50 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:33 [7386] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7386] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7386] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/125, version=0.155.576) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dd60 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dd60 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/825, version=0.155.576) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.576 2 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.577 (null) Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=577 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.18 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:33 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.577 to 0.155.576 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/827, version=0.155.577) Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/127) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 125 for root_free=11685 passed Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11685 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11685, Current: 11685, Stored: 11685 Feb 17 18:40:33 [7386] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11685 for localhost Feb 17 18:40:33 [7386] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7386-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7386-11) state:2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7386-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7386-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7386-11-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/127, version=0.155.577) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 127 for cpu_load=0.68, passed Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:33 [7398] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7398 id=54d82df4-4374-481c-8705-0d27258c6424 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7398-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7398] Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7398] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7398] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7398] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7398] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9067 for localhost Feb 17 18:40:33 [7398] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9067 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9067, Current: 9069, Stored: 9069 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9067 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9067) Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/129) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 129: var_log_free=9067 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7398-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7398-11) state:2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7398-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7398-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7398-11-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [7410] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df40 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.577 2 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.578 (null) Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=578 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-ram_free']: @value=600 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/829, version=0.155.578) Feb 17 18:40:33 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.578 to 0.155.577 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df40 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.578 2 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.579 (null) Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=579 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9067 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/129, version=0.155.579) Feb 17 18:40:33 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.579 to 0.155.578 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/831, version=0.155.579) Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[19]) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 129 for var_log_free=9067 passed Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/131) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7410 id=1a3697b6-6bd5-400f-8276-bf8c1a97b5ec Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7410-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7410] Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [7410] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7410] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [7410] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/131, version=0.155.579) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18422, Stored: 18422 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_lib_mysql_free is 18424 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_lib_mysql_free (18424) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e030 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [7410] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:40:33 [7410] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_lib_mysql_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[22]) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 133: var_lib_mysql_free=18424 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 131 for ram_free=250 passed Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7410-11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7410-11) state:2 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7410-11-header Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(6983) = 0: Resource temporarily unavailable (11) Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7079) = 0: Resource temporarily unavailable (11) Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7410-11-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7410-11-header Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7253 - exited with rc=0 Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7253:stderr [ -- empty -- ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7253:stdout [ arch: x86_64 ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7253:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7253:stdout [ free_swap: 3100 ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7253:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7253:stdout [ cpu_speed: 6999.99 ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7253:stdout [ cpu_cores: 2 ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7253:stdout [ cpu_load: 0.68, ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7253:stdout [ ram_total: 3050 ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7253:stdout [ ram_free: 250 ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7253:stdout [ root_free: 11685 ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7253:stdout [ root_free: 11685 ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7253:stdout [ var_log_free: 9067 ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7253:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:7253 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/133) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e030 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349eac0 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.579 2 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.580 (null) Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=580 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_lib_mysql_free']: @value=18424 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/133, version=0.155.580) Feb 17 18:40:33 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.580 to 0.155.579 Feb 17 18:40:33 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 133 for var_lib_mysql_free=18424 passed Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349eac0 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:33 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/833, version=0.155.580) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-mysql-wss: 2016/02/17_18:40:33 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:40:33 INFO: MySQL is not running Feb 17 18:40:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:33 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dff0 Feb 17 18:40:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:34 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.580 2 Feb 17 18:40:34 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.581 (null) Feb 17 18:40:34 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=581 Feb 17 18:40:34 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9065 Feb 17 18:40:34 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/835, version=0.155.581) Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:40:34 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:34 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.581 to 0.155.580 Feb 17 18:40:34 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/135) Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dff0 Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:34 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:34 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/135, version=0.155.581) Feb 17 18:40:34 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 135 for var_log_free=9067 passed ocf-ns_haproxy: 2016/02/17_18:40:34 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_haproxy: 2016/02/17_18:40:34 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:40:34 INFO: haproxy daemon running Feb 17 18:40:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(6983) = 0: Resource temporarily unavailable (11) Feb 17 18:40:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7079) = 0: Resource temporarily unavailable (11) Feb 17 18:40:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:7426 - exited with rc=0 Feb 17 18:40:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:7426:stderr [ -- empty -- ] Feb 17 18:40:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:7426:stdout [ -- empty -- ] Feb 17 18:40:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:7426 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:40:35 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.611ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.852ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.547ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.717ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:40:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:40:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:6983 - exited with rc=0 Feb 17 18:40:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:6983:stderr [ -- empty -- ] Feb 17 18:40:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:6983:stdout [ -- empty -- ] Feb 17 18:40:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:6983 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7079) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:40:35 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.647ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.530ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.543ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:40:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:7079 - exited with rc=0 Feb 17 18:40:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:7079:stderr [ -- empty -- ] Feb 17 18:40:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:7079:stdout [ -- empty -- ] Feb 17 18:40:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:7079 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:40:36 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:40:36 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:40:36 INFO: MySQL is not running Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:39 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:39 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:39 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 0af65dd173a01d058f0acd9ba88c4705 for 0.155.581 (0x1733cf0 0) Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:39 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-mysql-wss: 2016/02/17_18:40:39 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:40:39 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:40:39 INFO: MySQL is not running Feb 17 18:40:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:40:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:40:42 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:40:42 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:40:42 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:40:43 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.791ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.827ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.548ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.550ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:40:43 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:40:43 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:7590 - exited with rc=0 Feb 17 18:40:43 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:7590:stderr [ -- empty -- ] Feb 17 18:40:43 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:7590:stdout [ -- empty -- ] Feb 17 18:40:43 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:7590 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:43 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7609) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:40:43 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.752ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.527ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.536ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:40:43 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:43 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:7609 - exited with rc=0 Feb 17 18:40:43 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:7609:stderr [ -- empty -- ] Feb 17 18:40:43 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:7609:stdout [ -- empty -- ] Feb 17 18:40:43 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:7609 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:40:45 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:40:45 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:40:45 INFO: MySQL is not running Feb 17 18:40:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:40:47 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff Feb 17 18:40:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:7829 - exited with rc=8 Feb 17 18:40:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:7829:stderr [ -- empty -- ] Feb 17 18:40:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:7829:stdout [ vrouter ] Feb 17 18:40:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:7829:stdout [ mgmt-conntrd ] Feb 17 18:40:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:7829 exit-code:8 exec-time:0ms queue-time:0ms Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:40:48 INFO: ntp daemon running Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:7859 - exited with rc=0 Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:7859:stderr [ -- empty -- ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:7859:stdout [ -- empty -- ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:7859 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:40:48 [7916] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7916 id=477b2bd7-98ac-46f8-a363-d2d3c2a3145a Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7916-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7916] Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7916] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7916] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7916] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:40:48 [7916] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:40:48 [7916] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7916-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7916-11) state:2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7916-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7916-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7916-11-header Feb 17 18:40:48 [7919] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7919 id=fd5bae7d-6b63-4a22-b644-3e3c67cf4fdb Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7919-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7919] Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7919] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7919] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7919] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:40:48 [7919] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:40:48 [7919] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7919-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7919-11) state:2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7919-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7919-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7919-11-header Feb 17 18:40:48 [7933] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7933 id=e5c3c4c6-3c65-4538-8abd-bc77698eae32 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7933-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7933] Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7933] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7933] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7933] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3100 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3100, Current: 3100, Stored: 3100 Feb 17 18:40:48 [7933] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3100 for localhost Feb 17 18:40:48 [7933] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7933-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7933-11) state:2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7933-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7933-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7933-11-header Feb 17 18:40:48 [7940] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7940 id=6fb9d513-2f7c-47f6-9f37-4a80708569ab Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7940-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7940] Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7940] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7940] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7940] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:40:48 [7940] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:40:48 [7940] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7940-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7940-11) state:2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7940-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7940-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7940-11-header Feb 17 18:40:48 [7941] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7941 id=cacc25d2-5965-4a6c-93f1-6dcef42d0783 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7941-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7941] Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7941] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7941] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7941] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:40:48 [7941] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:40:48 [7941] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7941-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7941-11) state:2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7941-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7941-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7941-11-header Feb 17 18:40:48 [7942] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7942 id=9420b5d3-3380-490f-a442-bea7ce0cce45 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7942-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7942] Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7942] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7942] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7942] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:40:48 [7942] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:40:48 [7942] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7942-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7942-11) state:2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7942-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7942-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7942-11-header Feb 17 18:40:48 [7947] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7947 id=56fa4454-1f51-40d3-90cc-e1e3708c2c22 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7947-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7947] Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7947] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7947] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7947] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.68, Feb 17 18:40:48 [7947] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.68, for localhost Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.68,, Current: 0.68,, Stored: 0.68, Feb 17 18:40:48 [7947] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7947-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7947-11) state:2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7947-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7947-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7947-11-header Feb 17 18:40:48 [7952] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7952 id=95781057-ecc7-45c8-bc87-400a6d032760 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7952-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7952] Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7952] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7952] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7952] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:40:48 [7952] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:40:48 [7952] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7952-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7952-11) state:2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7952-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7952-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7952-11-header Feb 17 18:40:48 [7957] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7957 id=89d9a972-9f02-4d35-8ca1-38e9b8d589e8 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7957-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7957] Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7957] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7957] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7957] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=200 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 200, Current: 250, Stored: 250 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of ram_free is 200 Feb 17 18:40:48 [7957] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=200 for localhost Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: ram_free (200) Feb 17 18:40:48 [7957] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[19]) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/137) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 137: ram_free=200 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7957-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7957-11) state:2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7957-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7957-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7957-11-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df60 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.581 2 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.582 (null) Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=582 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-ram_free']: @value=200 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/137, version=0.155.582) Feb 17 18:40:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.582 to 0.155.581 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 137 for ram_free=200 passed Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df60 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/837, version=0.155.582) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:48 [7969] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7969 id=6fce743b-d5fe-4fdd-9a38-da74cb0076ee Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7969-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7969] Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7969] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7969] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7969] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11688 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11688, Current: 11685, Stored: 11685 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of root_free is 11688 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: root_free (11688) Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 139: root_free=11688 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/139) Feb 17 18:40:48 [7969] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11688 for localhost Feb 17 18:40:48 [7969] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e4f0 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7969-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7969-11) state:2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7969-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7969-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7969-11-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.582 2 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.583 (null) Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=583 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-root_free']: @value=11688 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/139, version=0.155.583) Feb 17 18:40:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.583 to 0.155.582 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e4f0 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 139 for root_free=11688 passed Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/839, version=0.155.583) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:48 [7981] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7981 id=12022123-9030-4129-b139-a02750ddf37f Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7981-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7981] Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7981] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7981] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7981] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11688 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11688, Current: 11688, Stored: 11688 Feb 17 18:40:48 [7981] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11688 for localhost Feb 17 18:40:48 [7981] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7981-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7981-11) state:2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7981-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7981-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7981-11-header Feb 17 18:40:48 [7993] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=7993 id=7e08f4a3-8a45-4b90-980d-bf4ecbc5bd2a Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-7993-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [7993] Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7993] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7993] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [7993] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9066 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9066, Current: 9067, Stored: 9067 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9066 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9066) Feb 17 18:40:48 [7993] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9066 for localhost Feb 17 18:40:48 [7993] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 141: var_log_free=9066 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/141) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-7993-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-7993-11) state:2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-7993-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-7993-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-7993-11-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e170 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.583 2 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.584 (null) Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=584 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9066 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/141, version=0.155.584) Feb 17 18:40:48 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.584 to 0.155.583 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e170 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 141 for var_log_free=9066 passed Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:48 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:48 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/841, version=0.155.584) Feb 17 18:40:48 [8005] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:48 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=8005 id=615b0a8c-dd9e-4dac-b1a7-4516238a4a9f Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-8005-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [8005] Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [8005] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [8005] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [8005] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:40:48 [8005] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:40:48 [8005] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7870) = 0: Resource temporarily unavailable (11) Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7889) = 0: Resource temporarily unavailable (11) Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7908 - exited with rc=0 Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7908:stderr [ -- empty -- ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7908:stdout [ arch: x86_64 ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7908:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7908:stdout [ free_swap: 3100 ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7908:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7908:stdout [ cpu_speed: 6999.99 ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7908:stdout [ cpu_cores: 2 ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7908:stdout [ cpu_load: 0.68, ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7908:stdout [ ram_total: 3050 ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7908:stdout [ ram_free: 200 ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7908:stdout [ root_free: 11688 ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7908:stdout [ root_free: 11688 ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7908:stdout [ var_log_free: 9066 ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:7908:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:40:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:7908 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-8005-11) Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-8005-11) state:2 Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-8005-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-8005-11-header Feb 17 18:40:48 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-8005-11-header ocf-mysql-wss: 2016/02/17_18:40:48 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:40:48 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:40:48 INFO: MySQL is not running Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dfb0 Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.584 2 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.585 (null) Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=585 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-free_swap']: @value=3000 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/843, version=0.155.585) Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:49 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.585 to 0.155.584 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='free_swap'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[13]) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dfb0 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/143) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/143, version=0.155.585) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 143 for free_swap=3100 passed Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.585 2 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.586 (null) Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=586 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.20 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/845, version=0.155.586) Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/145) Feb 17 18:40:49 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.586 to 0.155.585 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/145, version=0.155.586) Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 145 for cpu_load=0.68, passed Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e050 Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.586 2 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.587 (null) Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=587 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-root_free']: @value=11689 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/847, version=0.155.587) Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:49 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.587 to 0.155.586 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/147) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e050 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/147, version=0.155.587) Feb 17 18:40:49 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 147 for root_free=11688 passed Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:40:50 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7870) = 0: Resource temporarily unavailable (11) Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7889) = 0: Resource temporarily unavailable (11) Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(8047) = 0: Resource temporarily unavailable (11) Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:8048 - exited with rc=0 Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:8048:stderr [ -- empty -- ] Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:8048:stdout [ -- empty -- ] Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:8048 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_dns: 2016/02/17_18:40:50 INFO: dnsmasq daemon running Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7870) = 0: Resource temporarily unavailable (11) Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7889) = 0: Resource temporarily unavailable (11) Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:8047 - exited with rc=0 Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:8047:stderr [ -- empty -- ] Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:8047:stdout [ -- empty -- ] Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:8047 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7870) = 0: Resource temporarily unavailable (11) Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7889) = 0: Resource temporarily unavailable (11) Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:8096 - exited with rc=0 Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:8096:stderr [ -- empty -- ] Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:8096:stdout [ -- empty -- ] Feb 17 18:40:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:8096 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:40:51 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.644ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.673ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.544ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.542ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:40:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:40:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:7870 - exited with rc=0 Feb 17 18:40:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:7870:stderr [ -- empty -- ] Feb 17 18:40:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:7870:stdout [ -- empty -- ] Feb 17 18:40:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:7870 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(7889) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:40:51 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.793ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.549ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.538ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:40:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:7889 - exited with rc=0 Feb 17 18:40:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:7889:stderr [ -- empty -- ] Feb 17 18:40:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:7889:stdout [ -- empty -- ] Feb 17 18:40:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:7889 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:40:51 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:40:51 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:40:51 INFO: MySQL is not running Feb 17 18:40:53 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.003000 (9 ticks in 30s) Feb 17 18:40:53 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.800000 (full: 0.80 0.67 0.38 2/477 8198) Feb 17 18:40:53 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:40:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:40:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:8211 - exited with rc=0 Feb 17 18:40:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:8211:stderr [ -- empty -- ] Feb 17 18:40:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:8211:stdout [ -- empty -- ] Feb 17 18:40:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:8211 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:40:53 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:40:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:8236 - exited with rc=0 Feb 17 18:40:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:8236:stderr [ -- empty -- ] Feb 17 18:40:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:8236:stdout [ -- empty -- ] Feb 17 18:40:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:8236 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:40:54 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 367675f5a52d1fc469302c3dbd9a6ce8 for 0.155.587 (0x1733cf0 0) Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:40:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:40:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:40:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:40:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-mysql-wss: 2016/02/17_18:40:54 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:40:54 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:40:54 INFO: MySQL is not running Feb 17 18:40:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:40:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:40:57 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:40:58 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:40:58 INFO: MySQL is not running Feb 17 18:40:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:40:58 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:40:58 INFO: net.ipv4.ip_forward = 1 Feb 17 18:40:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(8342) = 0: Resource temporarily unavailable (11) Feb 17 18:40:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(8361) = 0: Resource temporarily unavailable (11) Feb 17 18:40:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:8430 - exited with rc=0 Feb 17 18:40:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:8430:stderr [ -- empty -- ] Feb 17 18:40:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:8430:stdout [ -- empty -- ] Feb 17 18:40:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:8430 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:40:59 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.751ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.768ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.539ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.525ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:40:59 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:40:59 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:8342 - exited with rc=0 Feb 17 18:40:59 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:8342:stderr [ -- empty -- ] Feb 17 18:40:59 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:8342:stdout [ -- empty -- ] Feb 17 18:40:59 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:8342 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:40:59 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(8361) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:40:59 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.930ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.537ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.526ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:40:59 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:40:59 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:8361 - exited with rc=0 Feb 17 18:40:59 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:8361:stderr [ -- empty -- ] Feb 17 18:40:59 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:8361:stdout [ -- empty -- ] Feb 17 18:40:59 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:8361 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:41:01 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:01 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:01 INFO: MySQL is not running Feb 17 18:41:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:41:03 [8843] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=8843 id=4f33f081-ae7d-4002-8725-04277dbb2a5e Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-8843-11) Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [8843] Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [8843] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [8843] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [8843] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:41:03 [8843] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:41:03 [8843] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-8843-11) Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-8843-11) state:2 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-8843-11-header Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-8843-11-header Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-8843-11-header Feb 17 18:41:03 [8846] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=8846 id=863b58d7-93c9-44c6-acad-67dbd5758a64 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-8846-11) Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [8846] Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [8846] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [8846] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [8846] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:41:03 [8846] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:41:03 [8846] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-8846-11) Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-8846-11) state:2 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-8846-11-header Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-8846-11-header Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-8846-11-header Feb 17 18:41:03 [8861] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=8861 id=ec0d3309-9fb6-4075-bf50-f97d5872b57b Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-8861-11) Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [8861] Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [8861] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [8861] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [8861] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [8861] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3100 for localhost Feb 17 18:41:03 [8861] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3100 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3100, Current: 3100, Stored: 3100 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-8861-11) Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-8861-11) state:2 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-8861-11-header Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-8861-11-header Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-8861-11-header Feb 17 18:41:03 [8868] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=8868 id=dfdff07c-664b-40bd-a68e-1918ca3edf05 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-8868-11) Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [8868] Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [8868] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [8868] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [8868] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:41:03 [8868] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:41:03 [8868] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-8868-11) Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-8868-11) state:2 Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-8868-11-header Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-8868-11-header Feb 17 18:41:03 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-8868-11-header Feb 17 18:41:04 [8873] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=8873 id=467d947c-150e-41b9-b601-cb607351118f Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-8873-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [8873] Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8873] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8873] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8873] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:41:04 [8873] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:41:04 [8873] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-8873-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-8873-11) state:2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-8873-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-8873-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-8873-11-header Feb 17 18:41:04 [8874] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=8874 id=4d6597b2-c6ca-4727-b2d4-a50a8159a602 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-8874-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [8874] Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8874] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8874] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8874] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:41:04 [8874] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:41:04 [8874] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-8874-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-8874-11) state:2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-8874-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-8874-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-8874-11-header ocf-mysql-wss: 2016/02/17_18:41:04 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:04 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:04 INFO: MySQL is not running Feb 17 18:41:04 [8884] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=8884 id=4737e057-2aeb-4580-95cb-ed43446ed71a Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-8884-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [8884] Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8884] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8884] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8884] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.68, Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.68,, Current: 0.68,, Stored: 0.68, Feb 17 18:41:04 [8884] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.68, for localhost Feb 17 18:41:04 [8884] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-8884-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-8884-11) state:2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-8884-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-8884-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-8884-11-header Feb 17 18:41:04 [8903] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=8903 id=7a57c7d1-f97a-42fd-919e-1fdabdb7cd5a Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-8903-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [8903] Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8903] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8903] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8903] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:41:04 [8903] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:41:04 [8903] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-8903-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-8903-11) state:2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-8903-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-8903-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-8903-11-header Feb 17 18:41:04 [8908] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=8908 id=0c2d08e0-9f79-4fe4-ab00-a811d05c343d Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-8908-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [8908] Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8908] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8908] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8908] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=200 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 200, Current: 200, Stored: 200 Feb 17 18:41:04 [8908] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=200 for localhost Feb 17 18:41:04 [8908] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-8908-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-8908-11) state:2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-8908-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-8908-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-8908-11-header Feb 17 18:41:04 [8921] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=8921 id=50f6509f-fbef-4fc0-b4b5-602e430e8d41 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-8921-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [8921] Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8921] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8921] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8921] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11693 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11693, Current: 11688, Stored: 11688 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of root_free is 11693 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: root_free (11693) Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 149: root_free=11693 Feb 17 18:41:04 [8921] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11693 for localhost Feb 17 18:41:04 [8921] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/149) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.587 2 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.588 (null) Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=588 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-root_free']: @value=11693 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/149, version=0.155.588) Feb 17 18:41:04 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.588 to 0.155.587 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 149 for root_free=11693 passed Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-8921-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-8921-11) state:2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-8921-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-8921-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-8921-11-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349eb20 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349eb20 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:04 [8934] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/849, version=0.155.588) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=8934 id=a177ac6e-4d36-4ce3-98de-8452a456cca8 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-8934-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [8934] Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8934] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8934] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8934] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11693 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11693, Current: 11693, Stored: 11693 Feb 17 18:41:04 [8934] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11693 for localhost Feb 17 18:41:04 [8934] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-8934-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-8934-11) state:2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-8934-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-8934-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-8934-11-header Feb 17 18:41:04 [8948] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=8948 id=67c9060f-4aa4-4c41-acdf-c54c2547526c Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-8948-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [8948] Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8948] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8948] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8948] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9066 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9066, Current: 9066, Stored: 9066 Feb 17 18:41:04 [8948] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9066 for localhost Feb 17 18:41:04 [8948] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-8948-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-8948-11) state:2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-8948-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-8948-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-8948-11-header ocf-ns_haproxy: 2016/02/17_18:41:04 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:41:04 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name Feb 17 18:41:04 [8977] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=8977 id=7511beba-d209-4b6a-8ab1-8ec85f104a71 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-8977-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [8977] Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8977] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8977] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [8977] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:41:04 [8977] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:41:04 [8977] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-8977-11) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-8977-11) state:2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-8977-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-8977-11-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-8977-11-header Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:8835 - exited with rc=0 Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:8835:stderr [ -- empty -- ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:8835:stdout [ arch: x86_64 ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:8835:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:8835:stdout [ free_swap: 3100 ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:8835:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:8835:stdout [ cpu_speed: 6999.99 ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:8835:stdout [ cpu_cores: 2 ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:8835:stdout [ cpu_load: 0.68, ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:8835:stdout [ ram_total: 3050 ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:8835:stdout [ ram_free: 200 ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:8835:stdout [ root_free: 11693 ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:8835:stdout [ root_free: 11693 ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:8835:stdout [ var_log_free: 9066 ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:8835:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:8835 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(8922) = 0: Resource temporarily unavailable (11) ocf-ns_haproxy: 2016/02/17_18:41:04 INFO: haproxy daemon running Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:8922 - exited with rc=0 Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:8922:stderr [ -- empty -- ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:8922:stdout [ -- empty -- ] Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:8922 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='free_swap'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[13]) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/151) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.588 2 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.589 (null) Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=589 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-free_swap']: @value=2950 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/851, version=0.155.589) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:04 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.589 to 0.155.588 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/151, version=0.155.589) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 151 for free_swap=3100 passed Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df10 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.589 2 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.590 (null) Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=590 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.19 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/853, version=0.155.590) Feb 17 18:41:04 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.590 to 0.155.589 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/153) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df10 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/153, version=0.155.590) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 153 for cpu_load=0.68, passed Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1cc0 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.590 2 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.591 (null) Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=591 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-root_free']: @value=11692 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/855, version=0.155.591) Feb 17 18:41:04 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.591 to 0.155.590 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/155) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1cc0 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/155, version=0.155.591) Feb 17 18:41:04 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 155 for root_free=11693 passed Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:41:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:41:07 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:07 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:07 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:41:07 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.607ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.641ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.551ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.531ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:41:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:41:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:9003 - exited with rc=0 Feb 17 18:41:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:9003:stderr [ -- empty -- ] Feb 17 18:41:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:9003:stdout [ -- empty -- ] Feb 17 18:41:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:9003 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9031) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:41:07 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.916ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.608ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:41:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:9031 - exited with rc=0 Feb 17 18:41:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:9031:stderr [ -- empty -- ] Feb 17 18:41:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:9031:stdout [ -- empty -- ] Feb 17 18:41:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:9031 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-metadata-agent_monitor_60000 Feb 17 18:41:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:9365 - exited with rc=0 Feb 17 18:41:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:9365:stderr [ -- empty -- ] Feb 17 18:41:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:9365:stdout [ -- empty -- ] Feb 17 18:41:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-metadata-agent action:monitor call_id:131 pid:9365 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:41:08 INFO: ntp daemon running Feb 17 18:41:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:9399 - exited with rc=0 Feb 17 18:41:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:9399:stderr [ -- empty -- ] Feb 17 18:41:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:9399:stdout [ -- empty -- ] Feb 17 18:41:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:9399 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:09 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:09 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:09 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 4b0a8b8f626d4c7bddecb8435be13c63 for 0.155.591 (0x1733cf0 0) Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:09 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:41:10 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:9557 - exited with rc=0 Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:9557:stderr [ -- empty -- ] Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:9557:stdout [ -- empty -- ] Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:9557 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 ocf-mysql-wss: 2016/02/17_18:41:10 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:10 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:10 INFO: MySQL is not running ocf-ns_dns: 2016/02/17_18:41:10 INFO: dnsmasq daemon running Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:9586 - exited with rc=0 Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:9586:stderr [ -- empty -- ] Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:9586:stdout [ -- empty -- ] Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:9586 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:9619 - exited with rc=0 Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:9619:stderr [ -- empty -- ] Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:9619:stdout [ -- empty -- ] Feb 17 18:41:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:9619 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:41:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:41:13 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:13 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:13 INFO: MySQL is not running Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9729) = 0: Resource temporarily unavailable (11) Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9748) = 0: Resource temporarily unavailable (11) Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:9819 - exited with rc=0 Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:9819:stderr [ -- empty -- ] Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:9819:stdout [ -- empty -- ] Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:9819 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:41:13 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9729) = 0: Resource temporarily unavailable (11) Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9748) = 0: Resource temporarily unavailable (11) Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:9849 - exited with rc=0 Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:9849:stderr [ -- empty -- ] Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:9849:stdout [ -- empty -- ] Feb 17 18:41:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:9849 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:41:14 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff Feb 17 18:41:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9729) = 0: Resource temporarily unavailable (11) Feb 17 18:41:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9748) = 0: Resource temporarily unavailable (11) Feb 17 18:41:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:9887 - exited with rc=8 Feb 17 18:41:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:9887:stderr [ -- empty -- ] Feb 17 18:41:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:9887:stdout [ vrouter ] Feb 17 18:41:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:9887:stdout [ mgmt-conntrd ] Feb 17 18:41:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:9887 exit-code:8 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:41:15 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.756ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.775ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.643ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.536ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:41:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:41:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:9729 - exited with rc=0 Feb 17 18:41:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:9729:stderr [ -- empty -- ] Feb 17 18:41:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:9729:stdout [ -- empty -- ] Feb 17 18:41:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:9729 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9748) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:41:15 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.651ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.543ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.537ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:41:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:9748 - exited with rc=0 Feb 17 18:41:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:9748:stderr [ -- empty -- ] Feb 17 18:41:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:9748:stdout [ -- empty -- ] Feb 17 18:41:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:9748 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:41:16 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:16 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:16 INFO: MySQL is not running ocf-mysql-wss: 2016/02/17_18:41:19 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:19 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:19 INFO: MySQL is not running Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:41:19 [10073] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10073 id=3cd611e8-6191-4384-a5e8-539526539094 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10073-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10073] Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10073] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10073] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10073] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:41:19 [10073] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:41:19 [10073] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10073-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10073-11) state:2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10073-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10073-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10073-11-header Feb 17 18:41:19 [10077] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10077 id=712cf228-f6fa-4597-9570-38625113f63a Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10077-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10077] Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10077] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10077] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10077] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:41:19 [10077] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:41:19 [10077] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10077-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10077-11) state:2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10077-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10077-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10077-11-header Feb 17 18:41:19 [10092] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10092 id=ac7aa606-d804-485b-b9c4-e4eb0c380db9 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10092-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10092] Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10092] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10092] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10092] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3100 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3100, Current: 3100, Stored: 3100 Feb 17 18:41:19 [10092] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3100 for localhost Feb 17 18:41:19 [10092] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10092-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10092-11) state:2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10092-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10092-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10092-11-header Feb 17 18:41:19 [10100] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10100 id=524b9643-b135-4381-8ee1-148ead959a0e Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10100-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10100] Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10100] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10100] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10100] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:41:19 [10100] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:41:19 [10100] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10100-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10100-11) state:2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10100-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10100-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10100-11-header Feb 17 18:41:19 [10102] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10102 id=660ef191-2860-458e-863d-4985b68e4f56 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10102-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10102] Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10102] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10102] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10102] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:41:19 [10102] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:41:19 [10102] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10102-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10102-11) state:2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10102-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10102-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10102-11-header Feb 17 18:41:19 [10104] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10104 id=f608e742-3982-415c-8709-c1f30a54a836 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10104-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10104] Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10104] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10104] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10104] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:41:19 [10104] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:41:19 [10104] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10104-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10104-11) state:2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10104-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10104-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10104-11-header Feb 17 18:41:19 [10109] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10109 id=147ae6ce-aa14-4c86-ab34-837f6ff2b886 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10109-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10109] Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10109] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10109] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10109] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.70, Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.70,, Current: 0.68,, Stored: 0.68, Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.70, Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.70,) Feb 17 18:41:19 [10109] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.70, for localhost Feb 17 18:41:19 [10109] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 157: cpu_load=0.70, Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/157) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10109-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10109-11) state:2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10109-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10109-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10109-11-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349def0 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.591 2 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.592 (null) Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=592 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.70, Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/157, version=0.155.592) Feb 17 18:41:19 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.592 to 0.155.591 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 157 for cpu_load=0.70, passed Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349def0 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/857, version=0.155.592) Feb 17 18:41:19 [10114] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10114 id=6d02447c-78a9-475d-9f5c-f76eedc0d330 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10114-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10114] Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10114] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:19 [10114] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:19 [10114] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:41:19 [10114] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:41:19 [10114] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10114-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10114-11) state:2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10114-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10114-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10114-11-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:19 [10120] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10120 id=63b96abf-dfe5-4666-b0da-a1029e6072b0 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10120-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10120] Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10120] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10120] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10120] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=300 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 300, Current: 200, Stored: 200 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of ram_free is 300 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: ram_free (300) Feb 17 18:41:19 [10120] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=300 for localhost Feb 17 18:41:19 [10120] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[19]) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 159: ram_free=300 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10120-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10120-11) state:2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10120-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10120-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10120-11-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/159) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1d00 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.592 2 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.593 (null) Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=593 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-ram_free']: @value=300 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/159, version=0.155.593) Feb 17 18:41:19 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.593 to 0.155.592 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 159 for ram_free=300 passed Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1d00 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/859, version=0.155.593) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:19 [10134] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10134 id=f85a334a-f5a9-4533-b9a5-e5354a200333 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10134-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10134] Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10134] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10134] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10134] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11693 Feb 17 18:41:19 [10134] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11693 for localhost Feb 17 18:41:19 [10134] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11693, Current: 11693, Stored: 11693 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10134-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10134-11) state:2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10134-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10134-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10134-11-header Feb 17 18:41:19 [10147] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10147 id=ee6c57a5-979c-43bc-bdbe-ced620d0dc3e Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10147-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10147] Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10147] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10147] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10147] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11693 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11693, Current: 11693, Stored: 11693 Feb 17 18:41:19 [10147] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11693 for localhost Feb 17 18:41:19 [10147] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10147-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10147-11) state:2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10147-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10147-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10147-11-header Feb 17 18:41:19 [10159] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10159 id=6937fc6e-19a5-4957-b601-b63db714069a Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10159-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10159] Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10159] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10159] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10159] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9065 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9065, Current: 9066, Stored: 9066 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9065 Feb 17 18:41:19 [10159] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9065 for localhost Feb 17 18:41:19 [10159] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9065) Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/161) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 161: var_log_free=9065 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10159-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10159-11) state:2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10159-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10159-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10159-11-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df50 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.593 2 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.594 (null) Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=594 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9065 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/161, version=0.155.594) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 161 for var_log_free=9065 passed Feb 17 18:41:19 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.594 to 0.155.593 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df50 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:19 [10171] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/861, version=0.155.594) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10171 id=4d73af2b-875b-42ae-9488-a80ab3debf8a Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10171-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10171] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:19 [10171] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10171] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [10171] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:41:19 [10171] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:41:19 [10171] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10171-11) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10171-11) state:2 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10171-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10171-11-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10171-11-header Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10064 - exited with rc=0 Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10064:stderr [ -- empty -- ] Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10064:stdout [ arch: x86_64 ] Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10064:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10064:stdout [ free_swap: 3100 ] Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10064:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10064:stdout [ cpu_speed: 6999.99 ] Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10064:stdout [ cpu_cores: 2 ] Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10064:stdout [ cpu_load: 0.70, ] Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10064:stdout [ ram_total: 3050 ] Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10064:stdout [ ram_free: 300 ] Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10064:stdout [ root_free: 11693 ] Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10064:stdout [ root_free: 11693 ] Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10064:stdout [ var_log_free: 9065 ] Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10064:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:41:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:10064 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df10 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.594 2 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.595 (null) Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=595 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-free_swap']: @value=2800 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/863, version=0.155.595) Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.595 to 0.155.594 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='free_swap'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[13]) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/163) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/163, version=0.155.595) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 163 for free_swap=3100 passed Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dd60 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.595 2 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.596 (null) Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=596 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.22 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/865, version=0.155.596) Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:19 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.596 to 0.155.595 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/165) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dd60 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/165, version=0.155.596) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 165 for cpu_load=0.70, passed Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349de50 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.596 2 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.597 (null) Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=597 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9064 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/867, version=0.155.597) Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:41:19 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.597 to 0.155.596 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349de50 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/167) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/167, version=0.155.597) Feb 17 18:41:19 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 167 for var_log_free=9065 passed Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:41:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:41:22 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:22 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:22 INFO: MySQL is not running Feb 17 18:41:23 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.001333 (4 ticks in 30s) Feb 17 18:41:23 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.960000 (full: 0.96 0.72 0.40 2/479 10279) Feb 17 18:41:23 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 ocf-ns_IPaddr2: 2016/02/17_18:41:23 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 1.111ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 1.136ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.550ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.544ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:41:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:41:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:10181 - exited with rc=0 Feb 17 18:41:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:10181:stderr [ -- empty -- ] Feb 17 18:41:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:10181:stdout [ -- empty -- ] Feb 17 18:41:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:10181 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10200) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:41:23 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.771ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.543ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.543ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:41:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:10200 - exited with rc=0 Feb 17 18:41:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:10200:stderr [ -- empty -- ] Feb 17 18:41:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:10200:stdout [ -- empty -- ] Feb 17 18:41:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:10200 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:24 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:24 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 7cc1435f293eceea47e41ffd28b9aacf for 0.155.597 (0x1733cf0 0) Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:24 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-mysql-wss: 2016/02/17_18:41:25 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:25 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:25 INFO: MySQL is not running ocf-mysql-wss: 2016/02/17_18:41:28 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:28 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:28 INFO: MySQL is not running Feb 17 18:41:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:41:28 INFO: ntp daemon running Feb 17 18:41:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:10415 - exited with rc=0 Feb 17 18:41:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:10415:stderr [ -- empty -- ] Feb 17 18:41:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:10415:stdout [ -- empty -- ] Feb 17 18:41:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:10415 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:41:28 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:41:28 INFO: net.ipv4.ip_forward = 1 Feb 17 18:41:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:10426 - exited with rc=0 Feb 17 18:41:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:10426:stderr [ -- empty -- ] Feb 17 18:41:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:10426:stdout [ -- empty -- ] Feb 17 18:41:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:10426 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:41:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:41:30 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10450) = 0: Resource temporarily unavailable (11) Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10469) = 0: Resource temporarily unavailable (11) Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:10633 - exited with rc=0 Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:10633:stderr [ -- empty -- ] Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:10633:stdout [ -- empty -- ] Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:10633 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 ocf-ns_dns: 2016/02/17_18:41:30 INFO: dnsmasq daemon running Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10450) = 0: Resource temporarily unavailable (11) Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10469) = 0: Resource temporarily unavailable (11) Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:10664 - exited with rc=0 Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:10664:stderr [ -- empty -- ] Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:10664:stdout [ -- empty -- ] Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:10664 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10450) = 0: Resource temporarily unavailable (11) Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10469) = 0: Resource temporarily unavailable (11) Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:10682 - exited with rc=0 Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:10682:stderr [ -- empty -- ] Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:10682:stdout [ -- empty -- ] Feb 17 18:41:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:10682 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:41:31 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:31 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:31 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:41:31 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.655ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.680ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.548ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.546ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:41:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:41:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:10450 - exited with rc=0 Feb 17 18:41:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:10450:stderr [ -- empty -- ] Feb 17 18:41:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:10450:stdout [ -- empty -- ] Feb 17 18:41:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:10450 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10469) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:41:32 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.682ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.567ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.531ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:41:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:10469 - exited with rc=0 Feb 17 18:41:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:10469:stderr [ -- empty -- ] Feb 17 18:41:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:10469:stdout [ -- empty -- ] Feb 17 18:41:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:10469 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:41:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 Feb 17 18:41:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:10825 - exited with rc=0 Feb 17 18:41:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:10825:stderr [ -- empty -- ] Feb 17 18:41:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:10825:stdout [ -- empty -- ] Feb 17 18:41:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:10825 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10850) = 0: Resource temporarily unavailable (11) ocf-neutron-dhcp-agent: 2016/02/17_18:41:33 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:41:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:10850 - exited with rc=0 Feb 17 18:41:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:10850:stderr [ -- empty -- ] Feb 17 18:41:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:10850:stdout [ -- empty -- ] Feb 17 18:41:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:10850 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-mysql-wss: 2016/02/17_18:41:34 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:34 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:34 INFO: MySQL is not running ocf-ns_haproxy: 2016/02/17_18:41:34 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:41:34 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:41:34 INFO: haproxy daemon running Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:10883 - exited with rc=0 Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:10883:stderr [ -- empty -- ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:10883:stdout [ -- empty -- ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:10883 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:41:34 [10939] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10939 id=13c5db92-cbe2-4248-843c-aed2ff6707b9 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10939-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10939] Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10939] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10939] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10939] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:41:34 [10939] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:41:34 [10939] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10939-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10939-11) state:2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10939-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10939-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10939-11-header Feb 17 18:41:34 [10942] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10942 id=95d9989d-fd06-47ea-8dbc-8166f3cf42f9 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10942-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10942] Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10942] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10942] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10942] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:41:34 [10942] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:41:34 [10942] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10942-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10942-11) state:2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10942-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10942-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10942-11-header Feb 17 18:41:34 [10956] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10956 id=cdfd2837-2962-40e7-86cc-b96af9f7d78d Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10956-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10956] Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10956] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10956] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10956] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3100 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3100, Current: 3100, Stored: 3100 Feb 17 18:41:34 [10956] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3100 for localhost Feb 17 18:41:34 [10956] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10956-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10956-11) state:2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10956-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10956-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10956-11-header Feb 17 18:41:34 [10963] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10963 id=c4674863-c0aa-4d55-87a5-0adf9e2284a8 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10963-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10963] Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10963] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10963] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10963] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:41:34 [10963] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:41:34 [10963] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10963-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10963-11) state:2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10963-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10963-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10963-11-header Feb 17 18:41:34 [10964] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10964 id=f269f272-02e4-4f3d-ada1-e1c62c22434f Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10964-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10964] Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10964] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10964] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10964] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:41:34 [10964] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:41:34 [10964] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10964-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10964-11) state:2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10964-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10964-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10964-11-header Feb 17 18:41:34 [10965] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10965 id=2524131f-6e18-43cf-9be3-963d688ab655 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10965-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10965] Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10965] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10965] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10965] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:41:34 [10965] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:41:34 [10965] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10965-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10965-11) state:2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10965-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10965-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10965-11-header Feb 17 18:41:34 [10970] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10970 id=f025a60c-b6a8-4ed2-82e5-4594529a7d2e Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10970-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10970] Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10970] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10970] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10970] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.70, Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.70,, Current: 0.70,, Stored: 0.70, Feb 17 18:41:34 [10970] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.70, for localhost Feb 17 18:41:34 [10970] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10970-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10970-11) state:2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10970-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10970-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10970-11-header Feb 17 18:41:34 [10975] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10975 id=8f875dd2-b8d9-4596-998f-ca6a43597c6c Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10975-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10975] Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10975] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10975] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10975] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:41:34 [10975] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:41:34 [10975] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10975-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10975-11) state:2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10975-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10975-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10975-11-header Feb 17 18:41:34 [10980] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10980 id=a9ab2528-438c-4625-96c5-b27863d4a0c0 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10980-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10980] Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10980] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10980] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10980] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=300 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 300, Current: 300, Stored: 300 Feb 17 18:41:34 [10980] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=300 for localhost Feb 17 18:41:34 [10980] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10980-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10980-11) state:2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10980-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10980-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10980-11-header Feb 17 18:41:34 [10992] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=10992 id=c185aaba-98ef-464a-ac75-cbefa10b0986 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-10992-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10992] Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10992] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10992] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [10992] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11693 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11693, Current: 11693, Stored: 11693 Feb 17 18:41:34 [10992] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11693 for localhost Feb 17 18:41:34 [10992] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-10992-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-10992-11) state:2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-10992-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-10992-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-10992-11-header Feb 17 18:41:34 [11004] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11004 id=91d44e9e-8681-46b8-afda-2824895a315a Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11004-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11004] Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [11004] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [11004] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [11004] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11693 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11693, Current: 11693, Stored: 11693 Feb 17 18:41:34 [11004] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11693 for localhost Feb 17 18:41:34 [11004] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11004-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11004-11) state:2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11004-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11004-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11004-11-header Feb 17 18:41:34 [11016] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11016 id=ab890f9e-5e9e-4cac-aa5a-389ab2ef8f76 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11016-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11016] Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [11016] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [11016] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [11016] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9065 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9065, Current: 9065, Stored: 9065 Feb 17 18:41:34 [11016] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9065 for localhost Feb 17 18:41:34 [11016] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11016-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11016-11) state:2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11016-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11016-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11016-11-header Feb 17 18:41:34 [11028] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11028 id=eb2ec2c9-a071-4019-9fe9-fa00e750dca6 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11028-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11028] Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [11028] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [11028] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [11028] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:41:34 [11028] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:41:34 [11028] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11028-11) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11028-11) state:2 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11028-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11028-11-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11028-11-header Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10931 - exited with rc=0 Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10931:stderr [ -- empty -- ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10931:stdout [ arch: x86_64 ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10931:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10931:stdout [ free_swap: 3100 ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10931:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10931:stdout [ cpu_speed: 6999.99 ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10931:stdout [ cpu_cores: 2 ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10931:stdout [ cpu_load: 0.70, ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10931:stdout [ ram_total: 3050 ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10931:stdout [ ram_free: 300 ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10931:stdout [ root_free: 11693 ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10931:stdout [ root_free: 11693 ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10931:stdout [ var_log_free: 9065 ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:10931:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:41:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:10931 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df40 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:34 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.597 2 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.598 (null) Feb 17 18:41:34 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=598 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-ram_free']: @value=650 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/869, version=0.155.598) Feb 17 18:41:34 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.598 to 0.155.597 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[19]) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:34 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/169) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df40 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:34 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/169, version=0.155.598) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 169 for ram_free=300 passed Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df20 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:34 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.598 2 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.599 (null) Feb 17 18:41:34 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=599 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9063 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/871, version=0.155.599) Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:34 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.599 to 0.155.598 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df20 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:41:34 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/171) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:34 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:34 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/171, version=0.155.599) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:34 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 171 for var_log_free=9065 passed Feb 17 18:41:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:41:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:41:37 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:37 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:37 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:41:39 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 1.185ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 1.223ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.541ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.551ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:41:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:41:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11087 - exited with rc=0 Feb 17 18:41:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11087:stderr [ -- empty -- ] Feb 17 18:41:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11087:stdout [ -- empty -- ] Feb 17 18:41:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:11087 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11114) = 0: Resource temporarily unavailable (11) Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:39 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:39 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:39 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 9821b6eed043eef31bcac7893862eca3 for 0.155.599 (0x1733cf0 0) Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:39 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_IPaddr2: 2016/02/17_18:41:40 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.809ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.534ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:41:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:11114 - exited with rc=0 Feb 17 18:41:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:11114:stderr [ -- empty -- ] Feb 17 18:41:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:11114:stdout [ -- empty -- ] Feb 17 18:41:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:11114 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:41:40 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:40 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:40 INFO: MySQL is not running Feb 17 18:41:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:41:41 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff Feb 17 18:41:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:11266 - exited with rc=8 Feb 17 18:41:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:11266:stderr [ -- empty -- ] Feb 17 18:41:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:11266:stdout [ vrouter ] Feb 17 18:41:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:11266:stdout [ mgmt-conntrd ] Feb 17 18:41:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:11266 exit-code:8 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:41:43 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:43 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:43 INFO: MySQL is not running Feb 17 18:41:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:41:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:41:46 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:46 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:46 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:41:47 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.750ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.778ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.531ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.535ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:41:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11346 - exited with rc=0 Feb 17 18:41:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11346:stderr [ -- empty -- ] Feb 17 18:41:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11346:stdout [ -- empty -- ] Feb 17 18:41:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:11346 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11373) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:41:48 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.819ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.535ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.526ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:41:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:11373 - exited with rc=0 Feb 17 18:41:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:11373:stderr [ -- empty -- ] Feb 17 18:41:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:11373:stdout [ -- empty -- ] Feb 17 18:41:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:11373 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:41:48 INFO: ntp daemon running Feb 17 18:41:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:11474 - exited with rc=0 Feb 17 18:41:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:11474:stderr [ -- empty -- ] Feb 17 18:41:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:11474:stdout [ -- empty -- ] Feb 17 18:41:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:11474 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:41:49 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:49 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:49 INFO: MySQL is not running Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:41:49 [11546] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11546 id=3e246452-fc18-4d0b-b7a2-15f877994d36 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11546-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11546] Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11546] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11546] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11546] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:41:49 [11546] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:41:49 [11546] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11546-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11546-11) state:2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11546-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11546-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11546-11-header Feb 17 18:41:49 [11549] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11549 id=5e30c05a-43b6-4af0-b8ed-2ea89182604e Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11549-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11549] Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11549] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11549] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11549] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:41:49 [11549] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:41:49 [11549] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11549-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11549-11) state:2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11549-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11549-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11549-11-header Feb 17 18:41:49 [11563] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11563 id=d63aed70-9353-4ac7-b6c8-f9d6fab93588 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11563-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11563] Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11563] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11563] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11563] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3100 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3100, Current: 3100, Stored: 3100 Feb 17 18:41:49 [11563] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3100 for localhost Feb 17 18:41:49 [11563] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11563-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11563-11) state:2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11563-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11563-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11563-11-header Feb 17 18:41:49 [11570] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11570 id=5c0961e9-3cd0-40be-8b36-7c53fc48c862 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11570-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11570] Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11570] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11570] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11570] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:41:49 [11570] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:41:49 [11570] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11570-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11570-11) state:2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11570-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11570-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11570-11-header Feb 17 18:41:49 [11571] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11571 id=cbb56b1d-bee4-4b3a-b362-89cc7079e409 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11571-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11571] Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11571] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11571] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11571] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:41:49 [11571] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:41:49 [11571] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11571-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11571-11) state:2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11571-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11571-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11571-11-header Feb 17 18:41:49 [11572] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11572 id=1115ebb4-0e5b-4a0b-a063-a6d7b0b10b98 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11572-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11572] Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11572] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11572] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11572] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:41:49 [11572] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:41:49 [11572] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11572-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11572-11) state:2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11572-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11572-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11572-11-header Feb 17 18:41:49 [11577] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11577 id=01cf25ff-3166-4faf-8793-19cd2829efaa Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11577-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11577] Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11577] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11577] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11577] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.69, Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.69,, Current: 0.70,, Stored: 0.70, Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.69, Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.69,) Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 173: cpu_load=0.69, Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/173) Feb 17 18:41:49 [11577] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.69, for localhost Feb 17 18:41:49 [11577] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11577-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11577-11) state:2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11577-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11577-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11577-11-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df20 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.599 2 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.600 (null) Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=600 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.69, Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/173, version=0.155.600) Feb 17 18:41:49 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.600 to 0.155.599 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df20 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 173 for cpu_load=0.69, passed Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [11582] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/873, version=0.155.600) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11582 id=da444f41-c241-49ee-8597-5d743059add0 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11582-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11582] Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11582] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11582] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11582] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:41:49 [11582] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:41:49 [11582] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11582-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11582-11) state:2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11582-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11582-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11582-11-header Feb 17 18:41:49 [11587] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11587 id=bbcfbce2-291a-4fad-b0bb-4a7e36644c98 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11587-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11587] Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11587] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11587] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11587] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=300 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 300, Current: 300, Stored: 300 Feb 17 18:41:49 [11587] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=300 for localhost Feb 17 18:41:49 [11587] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11587-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11587-11) state:2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11587-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11587-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11587-11-header Feb 17 18:41:49 [11599] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11599 id=721543fa-9490-471b-bbc3-fcbd29d47636 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11599-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11599] Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11599] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11599] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11599] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11693, Stored: 11693 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of root_free is 11692 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: root_free (11692) Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 175: root_free=11692 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/175) Feb 17 18:41:49 [11599] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:41:49 [11599] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11599-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11599-11) state:2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11599-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11599-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11599-11-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e4b0 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.600 2 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.601 (null) Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=601 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-root_free']: @value=11692 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/175, version=0.155.601) Feb 17 18:41:49 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.601 to 0.155.600 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 175 for root_free=11692 passed Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e4b0 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:41:49 [11611] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/875, version=0.155.601) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11611 id=9c25f3c8-3282-45ae-88a6-47dc1e4e4f63 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11611-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11611] Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:49 [11611] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11611] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11611] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11611] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:41:49 [11611] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11611-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11611-11) state:2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11611-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11611-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11611-11-header Feb 17 18:41:49 [11624] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11624 id=17bdcc06-c844-4ff8-ab43-ca8fa29b44f7 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11624-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11624] Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11624] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11624] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11624] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9064 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9064, Current: 9065, Stored: 9065 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9064 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9064) Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 177: var_log_free=9064 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/177) Feb 17 18:41:49 [11624] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9064 for localhost Feb 17 18:41:49 [11624] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b90 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.601 2 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.602 (null) Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=602 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9064 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/177, version=0.155.602) Feb 17 18:41:49 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.602 to 0.155.601 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 177 for var_log_free=9064 passed Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11624-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11624-11) state:2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11624-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11624-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11624-11-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b90 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/877, version=0.155.602) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:49 [11636] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=11636 id=772b4810-1058-4a5f-ade6-7a6f11807c08 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-11636-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [11636] Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11636] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11636] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [11636] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:41:49 [11636] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:41:49 [11636] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-11636-11) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-11636-11) state:2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-11636-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-11636-11-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-11636-11-header Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:11538 - exited with rc=0 Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:11538:stderr [ -- empty -- ] Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:11538:stdout [ arch: x86_64 ] Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:11538:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:11538:stdout [ free_swap: 3100 ] Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:11538:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:11538:stdout [ cpu_speed: 6999.99 ] Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:11538:stdout [ cpu_cores: 2 ] Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:11538:stdout [ cpu_load: 0.69, ] Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:11538:stdout [ ram_total: 3050 ] Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:11538:stdout [ ram_free: 300 ] Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:11538:stdout [ root_free: 11692 ] Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:11538:stdout [ root_free: 11692 ] Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:11538:stdout [ var_log_free: 9064 ] Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:11538:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:41:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:11538 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df10 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.602 2 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.603 (null) Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=603 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-ram_free']: @value=600 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/879, version=0.155.603) Feb 17 18:41:49 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.603 to 0.155.602 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[19]) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/179) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df10 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:49 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/179, version=0.155.603) Feb 17 18:41:49 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 179 for ram_free=300 passed Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:49 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:41:50 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:11637 - exited with rc=0 Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:11637:stderr [ -- empty -- ] Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:11637:stdout [ -- empty -- ] Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:11637 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 ocf-ns_dns: 2016/02/17_18:41:50 INFO: dnsmasq daemon running Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:11664 - exited with rc=0 Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:11664:stderr [ -- empty -- ] Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:11664:stdout [ -- empty -- ] Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:11664 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:11686 - exited with rc=0 Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:11686:stderr [ -- empty -- ] Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:11686:stdout [ -- empty -- ] Feb 17 18:41:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:11686 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:41:52 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:52 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:52 INFO: MySQL is not running Feb 17 18:41:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:41:53 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.001000 (3 ticks in 30s) Feb 17 18:41:53 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.700000 (full: 0.70 0.68 0.40 2/476 11794) Feb 17 18:41:53 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11761) = 0: Resource temporarily unavailable (11) Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11798) = 0: Resource temporarily unavailable (11) Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:11827 - exited with rc=0 Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:11827:stderr [ -- empty -- ] Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:11827:stdout [ -- empty -- ] Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:11827 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:41:53 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11761) = 0: Resource temporarily unavailable (11) Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11798) = 0: Resource temporarily unavailable (11) Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:11852 - exited with rc=0 Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:11852:stderr [ -- empty -- ] Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:11852:stdout [ -- empty -- ] Feb 17 18:41:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:11852 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:41:54 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 02f76a923e7a29f69fd73cc7f347a108 for 0.155.603 (0x1733cf0 0) Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:41:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:41:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:41:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:41:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-mysql-wss: 2016/02/17_18:41:55 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:55 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:55 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:41:55 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.598ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.617ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.541ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.529ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:41:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:41:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11761 - exited with rc=0 Feb 17 18:41:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11761:stderr [ -- empty -- ] Feb 17 18:41:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:11761:stdout [ -- empty -- ] Feb 17 18:41:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:11761 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:41:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(11798) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:41:56 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.946ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.530ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:41:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:11798 - exited with rc=0 Feb 17 18:41:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:11798:stderr [ -- empty -- ] Feb 17 18:41:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:11798:stdout [ -- empty -- ] Feb 17 18:41:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:11798 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:41:58 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:41:58 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:41:58 INFO: MySQL is not running Feb 17 18:41:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:41:58 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:41:58 INFO: net.ipv4.ip_forward = 1 Feb 17 18:41:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:41:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:12018 - exited with rc=0 Feb 17 18:41:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:12018:stderr [ -- empty -- ] Feb 17 18:41:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:12018:stdout [ -- empty -- ] Feb 17 18:41:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:12018 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:42:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:42:01 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:01 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:01 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:42:04 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 1.308ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 1.349ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.541ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.540ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:12075 - exited with rc=0 Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:12075:stderr [ -- empty -- ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:12075:stdout [ -- empty -- ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:12075 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12113) = 0: Resource temporarily unavailable (11) Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:42:04 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:42:04 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_IPaddr2: 2016/02/17_18:42:04 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 8.494ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.540ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:12113 - exited with rc=0 Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:12113:stderr [ -- empty -- ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:12113:stdout [ -- empty -- ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:12113 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12209) = 0: Resource temporarily unavailable (11) ocf-ns_haproxy: 2016/02/17_18:42:04 INFO: haproxy daemon running Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:12209 - exited with rc=0 Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:12209:stderr [ -- empty -- ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:12209:stdout [ -- empty -- ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:12209 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:42:04 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:04 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:04 INFO: MySQL is not running Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:42:04 [12272] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=12272 id=7afa2153-2f36-48e3-9c2e-724d5ed270ae Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12272-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12272] Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12272] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12272] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12272] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:42:04 [12272] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:42:04 [12272] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12272-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12272-11) state:2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12272-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12272-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12272-11-header Feb 17 18:42:04 [12275] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=12275 id=c9937a77-f1ac-49bd-8e6e-7be7bc6d9629 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12275-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12275] Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12275] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12275] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12275] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:42:04 [12275] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:42:04 [12275] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12275-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12275-11) state:2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12275-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12275-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12275-11-header Feb 17 18:42:04 [12289] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=12289 id=516018dd-dbcf-4376-ae1c-5d18990ae709 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12289-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12289] Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12289] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12289] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12289] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3100 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3100, Current: 3100, Stored: 3100 Feb 17 18:42:04 [12289] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3100 for localhost Feb 17 18:42:04 [12289] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12289-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12289-11) state:2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12289-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12289-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12289-11-header Feb 17 18:42:04 [12296] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=12296 id=e82621ac-070d-41cd-a182-dd1d83286646 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12296-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12296] Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12296] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12296] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12296] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:42:04 [12296] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:42:04 [12296] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12296-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12296-11) state:2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12296-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12296-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12296-11-header Feb 17 18:42:04 [12297] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=12297 id=d9f7f641-cc4e-4814-9cc5-d5f6fe6b1d09 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12297-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12297] Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12297] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12297] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12297] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:42:04 [12297] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:42:04 [12297] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12297-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12297-11) state:2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12297-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12297-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12297-11-header Feb 17 18:42:04 [12298] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=12298 id=f0f96a78-e69b-4cb5-b704-0015b4945361 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12298-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12298] Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12298] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12298] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12298] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:42:04 [12298] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:42:04 [12298] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12298-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12298-11) state:2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12298-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12298-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12298-11-header Feb 17 18:42:04 [12303] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=12303 id=d548a252-d636-4aca-8c9c-5ff5969594e5 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12303-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12303] Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12303] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12303] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12303] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.66, Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.66,, Current: 0.69,, Stored: 0.69, Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.66, Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.66,) Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:42:04 [12303] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.66, for localhost Feb 17 18:42:04 [12303] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 181: cpu_load=0.66, Feb 17 18:42:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/181) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12303-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12303-11) state:2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12303-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12303-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12303-11-header Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:04 [12308] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.603 2 Feb 17 18:42:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.604 (null) Feb 17 18:42:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=604 Feb 17 18:42:04 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.66, Feb 17 18:42:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/181, version=0.155.604) Feb 17 18:42:04 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.604 to 0.155.603 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dff0 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dff0 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:04 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:04 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/881, version=0.155.604) Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:04 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=12308 id=9443b3d0-e787-45f8-9b48-535dfecc5c0e Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12308-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12308] Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 181 for cpu_load=0.66, passed Feb 17 18:42:04 [12308] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12308] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12308] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:42:04 [12308] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:42:04 [12308] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12308-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12308-11) state:2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12308-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12308-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12308-11-header Feb 17 18:42:04 [12313] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=12313 id=626a4a48-c47d-45d4-9036-7822007c5cb7 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12313-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12313] Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12313] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12313] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12313] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=300 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 300, Current: 300, Stored: 300 Feb 17 18:42:04 [12313] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=300 for localhost Feb 17 18:42:04 [12313] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12313-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12313-11) state:2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12313-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12313-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12313-11-header Feb 17 18:42:04 [12325] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=12325 id=97c85a67-da56-4fa2-8819-09700293aa90 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12325-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12325] Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12325] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12325] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12325] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:42:04 [12325] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:42:04 [12325] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12325-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12325-11) state:2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12325-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12325-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12325-11-header Feb 17 18:42:04 [12338] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=12338 id=976f7633-7ef4-42bf-a4c0-a7495cb1f2f5 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12338-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12338] Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12338] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12338] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12338] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:42:04 [12338] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:42:04 [12338] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12338-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12338-11) state:2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12338-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12338-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12338-11-header Feb 17 18:42:04 [12353] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=12353 id=bd5f0fe3-5ed0-4fb7-81f3-46cd7cbffef5 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12353-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12353] Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12353] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12353] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12353] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9064 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9064, Current: 9064, Stored: 9064 Feb 17 18:42:04 [12353] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9064 for localhost Feb 17 18:42:04 [12353] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12353-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12353-11) state:2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12353-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12353-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12353-11-header Feb 17 18:42:04 [12366] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=12366 id=5b9c383c-97f3-41cb-b8c2-6f9dde7b12e8 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-12366-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12366] Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12366] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12366] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [12366] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:42:04 [12366] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:42:04 [12366] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-12366-11) Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-12366-11) state:2 Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-12366-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-12366-11-header Feb 17 18:42:04 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-12366-11-header Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:12264 - exited with rc=0 Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:12264:stderr [ -- empty -- ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:12264:stdout [ arch: x86_64 ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:12264:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:12264:stdout [ free_swap: 3100 ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:12264:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:12264:stdout [ cpu_speed: 6999.99 ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:12264:stdout [ cpu_cores: 2 ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:12264:stdout [ cpu_load: 0.66, ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:12264:stdout [ ram_total: 3050 ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:12264:stdout [ ram_free: 300 ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:12264:stdout [ root_free: 11692 ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:12264:stdout [ root_free: 11692 ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:12264:stdout [ var_log_free: 9064 ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:12264:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:42:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:12264 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.604 2 Feb 17 18:42:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.605 (null) Feb 17 18:42:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=605 Feb 17 18:42:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.23 Feb 17 18:42:05 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.605 to 0.155.604 Feb 17 18:42:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/883, version=0.155.605) Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:42:05 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:42:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/183) Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/183, version=0.155.605) Feb 17 18:42:05 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 183 for cpu_load=0.66, passed Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-mysql-wss: 2016/02/17_18:42:07 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:07 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:07 INFO: MySQL is not running Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-metadata-agent_monitor_60000 Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:12454 - exited with rc=0 Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:12454:stderr [ -- empty -- ] Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:12454:stdout [ -- empty -- ] Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-metadata-agent action:monitor call_id:131 pid:12454 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:42:08 INFO: ntp daemon running Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:12483 - exited with rc=0 Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:12483:stderr [ -- empty -- ] Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:12483:stdout [ -- empty -- ] Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:12483 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:42:08 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:12499 - exited with rc=8 Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:12499:stderr [ -- empty -- ] Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:12499:stdout [ vrouter ] Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:12499:stdout [ mgmt-conntrd ] Feb 17 18:42:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:12499 exit-code:8 exec-time:0ms queue-time:0ms Feb 17 18:42:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:42:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:10 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:10 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:10 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: cdba5000c4807da758103ef9e3572024 for 0.155.605 (0x1733cf0 0) Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:10 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-heat-engine: 2016/02/17_18:42:10 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12523) = 0: Resource temporarily unavailable (11) Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12563) = 0: Resource temporarily unavailable (11) Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:12584 - exited with rc=0 Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:12584:stderr [ -- empty -- ] Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:12584:stdout [ -- empty -- ] Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:12584 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 ocf-ns_dns: 2016/02/17_18:42:10 INFO: dnsmasq daemon running Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12523) = 0: Resource temporarily unavailable (11) Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12563) = 0: Resource temporarily unavailable (11) Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:12611 - exited with rc=0 Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:12611:stderr [ -- empty -- ] Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:12611:stdout [ -- empty -- ] Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:12611 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:42:10 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:10 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:10 INFO: MySQL is not running Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12523) = 0: Resource temporarily unavailable (11) Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12563) = 0: Resource temporarily unavailable (11) Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:12645 - exited with rc=0 Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:12645:stderr [ -- empty -- ] Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:12645:stdout [ -- empty -- ] Feb 17 18:42:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:12645 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:42:12 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.765ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.796ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.548ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.531ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:42:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:42:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:12523 - exited with rc=0 Feb 17 18:42:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:12523:stderr [ -- empty -- ] Feb 17 18:42:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:12523:stdout [ -- empty -- ] Feb 17 18:42:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:12523 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(12563) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:42:12 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.754ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.526ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.544ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:42:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:12563 - exited with rc=0 Feb 17 18:42:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:12563:stderr [ -- empty -- ] Feb 17 18:42:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:12563:stdout [ -- empty -- ] Feb 17 18:42:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:12563 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:42:13 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:13 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:13 INFO: MySQL is not running Feb 17 18:42:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:42:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:12894 - exited with rc=0 Feb 17 18:42:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:12894:stderr [ -- empty -- ] Feb 17 18:42:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:12894:stdout [ -- empty -- ] Feb 17 18:42:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:12894 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:42:13 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:42:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:12920 - exited with rc=0 Feb 17 18:42:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:12920:stderr [ -- empty -- ] Feb 17 18:42:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:12920:stdout [ -- empty -- ] Feb 17 18:42:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:12920 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:42:16 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:16 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:16 INFO: MySQL is not running Feb 17 18:42:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:42:17 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:42:19 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:19 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:19 INFO: MySQL is not running Feb 17 18:42:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:42:19 [13300] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=13300 id=e380b128-3eff-4e9b-a392-c4c1be79be42 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-13300-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13300] Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13300] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13300] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13300] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:42:19 [13300] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:42:19 [13300] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-13300-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-13300-11) state:2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-13300-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-13300-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-13300-11-header Feb 17 18:42:19 [13303] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=13303 id=ca0020a9-5935-41a1-846e-4024fe51b371 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-13303-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13303] Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13303] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13303] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13303] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:42:19 [13303] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:42:19 [13303] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-13303-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-13303-11) state:2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-13303-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-13303-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-13303-11-header Feb 17 18:42:19 [13317] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=13317 id=87142a99-56e6-4936-a446-997eb85c39eb Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-13317-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13317] Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13317] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13317] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13317] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3050 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3100, Stored: 3100 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of free_swap is 3050 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: free_swap (3050) Feb 17 18:42:19 [13317] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3050 for localhost Feb 17 18:42:19 [13317] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='free_swap'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[13]) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 185: free_swap=3050 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-13317-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-13317-11) state:2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-13317-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-13317-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-13317-11-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/185) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dd80 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.605 2 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.606 (null) Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=606 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-free_swap']: @value=3050 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/185, version=0.155.606) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 185 for free_swap=3050 passed Feb 17 18:42:19 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.606 to 0.155.605 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dd80 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:19 [13324] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/885, version=0.155.606) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=13324 id=fb651e1d-b501-4cbd-b6a7-7f663fa1ef21 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-13324-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13324] Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13324] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13324] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13324] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:42:19 [13324] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:42:19 [13324] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:19 [13325] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201c130 for uid=0 gid=0 pid=13325 id=c666c108-3df0-4131-b5de-c19e3821b8b7 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-13325-12) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13325] Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-13324-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-13324-11) state:2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-13324-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-13324-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-13324-11-header Feb 17 18:42:19 [13325] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13325] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13325] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:42:19 [13325] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:42:19 [13325] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-13325-12) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-13325-12) state:2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-13325-12-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-13325-12-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-13325-12-header Feb 17 18:42:19 [13326] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=13326 id=ad1c653e-45bf-4d67-b331-83322a4f5331 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-13326-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13326] Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13326] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13326] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13326] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:42:19 [13326] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:42:19 [13326] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-13326-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-13326-11) state:2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-13326-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-13326-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-13326-11-header Feb 17 18:42:19 [13331] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=13331 id=e4209033-cca7-4ea8-b77a-9b8ef789aaa4 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-13331-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13331] Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13331] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13331] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13331] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.70, Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.70,, Current: 0.66,, Stored: 0.66, Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.70, Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.70,) Feb 17 18:42:19 [13331] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.70, for localhost Feb 17 18:42:19 [13331] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/187) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 187: cpu_load=0.70, Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-13331-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-13331-11) state:2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-13331-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-13331-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-13331-11-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e310 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.606 2 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.607 (null) Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=607 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.70, Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/187, version=0.155.607) Feb 17 18:42:19 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.607 to 0.155.606 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e310 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 187 for cpu_load=0.70, passed Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/887, version=0.155.607) Feb 17 18:42:19 [13336] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=13336 id=1c6a753e-f61a-4bd8-a240-8dfd9757aef0 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-13336-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13336] Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13336] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13336] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13336] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13336] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:42:19 [13336] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-13336-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-13336-11) state:2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-13336-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-13336-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-13336-11-header Feb 17 18:42:19 [13341] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=13341 id=afd3ca67-3129-4c52-a478-9db192692cc2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-13341-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13341] Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13341] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13341] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13341] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=500 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 500, Current: 300, Stored: 300 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of ram_free is 500 Feb 17 18:42:19 [13341] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=500 for localhost Feb 17 18:42:19 [13341] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: ram_free (500) Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[19]) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/189) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 189: ram_free=500 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-13341-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-13341-11) state:2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-13341-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-13341-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-13341-11-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df80 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:19 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.607 2 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.608 (null) Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=608 Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-ram_free']: @value=500 Feb 17 18:42:19 [13353] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:19 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/189, version=0.155.608) Feb 17 18:42:19 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.608 to 0.155.607 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 189 for ram_free=500 passed Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=13353 id=a82fb184-3f87-4ffe-b205-660ccb20d729 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-13353-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13353] Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13353] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13353] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [13353] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df80 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:42:19 [13353] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:42:19 [13353] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-13353-11) Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-13353-11) state:2 Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-13353-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-13353-11-header Feb 17 18:42:19 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-13353-11-header Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:19 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/889, version=0.155.608) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:20 [13365] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=13365 id=1d3ef37d-4927-48c7-a059-7ed160297894 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-13365-11) Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13365] Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [13365] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [13365] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [13365] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [13365] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:42:20 [13365] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-13365-11) Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-13365-11) state:2 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-13365-11-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-13365-11-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-13365-11-header Feb 17 18:42:20 [13377] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=13377 id=3414cf1f-17c9-4a9b-b5df-409c3c6c9b97 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-13377-11) Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13377] Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [13377] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [13377] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [13377] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9063 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9063, Current: 9064, Stored: 9064 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9063 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9063) Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 191: var_log_free=9063 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/191) Feb 17 18:42:20 [13377] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9063 for localhost Feb 17 18:42:20 [13377] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-13377-11) Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-13377-11) state:2 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-13377-11-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-13377-11-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-13377-11-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [13389] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df50 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.608 2 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.609 (null) Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=609 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9063 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/191, version=0.155.609) Feb 17 18:42:20 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.609 to 0.155.608 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df50 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 191 for var_log_free=9063 passed Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=13389 id=89e23749-72b7-4637-9f6c-06e5468f280b Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-13389-11) Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13389] Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:20 [13389] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [13389] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [13389] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:42:20 [13389] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:42:20 [13389] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-13389-11) Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-13389-11) state:2 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-13389-11-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-13389-11-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-13389-11-header Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13186) = 0: Resource temporarily unavailable (11) Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13223) = 0: Resource temporarily unavailable (11) Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13292 - exited with rc=0 Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13292:stderr [ -- empty -- ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13292:stdout [ arch: x86_64 ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13292:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13292:stdout [ free_swap: 3050 ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13292:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13292:stdout [ cpu_speed: 6999.99 ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13292:stdout [ cpu_cores: 2 ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13292:stdout [ cpu_load: 0.70, ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13292:stdout [ ram_total: 3050 ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13292:stdout [ ram_free: 500 ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13292:stdout [ root_free: 11692 ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13292:stdout [ root_free: 11692 ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13292:stdout [ var_log_free: 9063 ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13292:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:13292 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/891, version=0.155.609) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_IPaddr2: 2016/02/17_18:42:20 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.792ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.832ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.532ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.529ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:13186 - exited with rc=0 Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:13186:stderr [ -- empty -- ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:13186:stdout [ -- empty -- ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:13186 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13223) = 0: Resource temporarily unavailable (11) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df10 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.609 2 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.610 (null) Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=610 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-free_swap']: @value=2750 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/893, version=0.155.610) Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='free_swap'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[13]) Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:42:20 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.610 to 0.155.609 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/193) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df10 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/193, version=0.155.610) Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 193 for free_swap=3050 passed Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.610 2 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.611 (null) Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=611 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.25 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df40 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/895, version=0.155.611) Feb 17 18:42:20 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.611 to 0.155.610 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/195) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df40 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/195, version=0.155.611) Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 195 for cpu_load=0.70, passed Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349eb00 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.611 2 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.612 (null) Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=612 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-ram_free']: @value=650 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/897, version=0.155.612) Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[19]) Feb 17 18:42:20 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.612 to 0.155.611 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/197) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349eb00 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/197, version=0.155.612) Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 197 for ram_free=500 passed Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.612 2 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.613 (null) Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=613 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9062 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/899, version=0.155.613) Feb 17 18:42:20 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.613 to 0.155.612 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dd60 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/199) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dd60 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/199, version=0.155.613) Feb 17 18:42:20 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 199 for var_log_free=9063 passed Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_IPaddr2: 2016/02/17_18:42:20 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 1.193ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.551ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:13223 - exited with rc=0 Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:13223:stderr [ -- empty -- ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:13223:stdout [ -- empty -- ] Feb 17 18:42:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:13223 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:42:22 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:22 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:22 INFO: MySQL is not running Feb 17 18:42:23 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.001333 (4 ticks in 30s) Feb 17 18:42:23 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.760000 (full: 0.76 0.70 0.42 2/480 13475) Feb 17 18:42:23 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:42:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:25 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: c650fece70bb292e9df8ec65ea37ad49 for 0.155.613 (0x1733cf0 0) Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:42:25 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:25 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:25 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:42:28 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.682ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.724ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.544ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.541ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:13512 - exited with rc=0 Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:13512:stderr [ -- empty -- ] Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:13512:stdout [ -- empty -- ] Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:13512 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13541) = 0: Resource temporarily unavailable (11) Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:42:28 INFO: ntp daemon running Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13541) = 0: Resource temporarily unavailable (11) Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:13619 - exited with rc=0 Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:13619:stderr [ -- empty -- ] Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:13619:stdout [ -- empty -- ] Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:13619 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:42:28 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-mysql-wss: 2016/02/17_18:42:28 ERROR: Not enough arguments [1] to ocf_log. ocf-ns_vrouter: 2016/02/17_18:42:28 INFO: net.ipv4.ip_forward = 1 Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13541) = 0: Resource temporarily unavailable (11) Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:13634 - exited with rc=0 Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:13634:stderr [ -- empty -- ] Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:13634:stdout [ -- empty -- ] Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:13634 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:42:28 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:28 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:42:28 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.705ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.538ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.550ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:13541 - exited with rc=0 Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:13541:stderr [ -- empty -- ] Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:13541:stdout [ -- empty -- ] Feb 17 18:42:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:13541 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:42:30 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:13706 - exited with rc=0 Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:13706:stderr [ -- empty -- ] Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:13706:stdout [ -- empty -- ] Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:13706 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 ocf-ns_dns: 2016/02/17_18:42:30 INFO: dnsmasq daemon running Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:13733 - exited with rc=0 Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:13733:stderr [ -- empty -- ] Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:13733:stdout [ -- empty -- ] Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:13733 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:13751 - exited with rc=0 Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:13751:stderr [ -- empty -- ] Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:13751:stdout [ -- empty -- ] Feb 17 18:42:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:13751 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:42:31 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:31 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:31 INFO: MySQL is not running Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13852) = 0: Resource temporarily unavailable (11) Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13877) = 0: Resource temporarily unavailable (11) Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:13896 - exited with rc=0 Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:13896:stderr [ -- empty -- ] Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:13896:stdout [ -- empty -- ] Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:13896 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:42:33 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13852) = 0: Resource temporarily unavailable (11) Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13877) = 0: Resource temporarily unavailable (11) Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:13921 - exited with rc=0 Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:13921:stderr [ -- empty -- ] Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:13921:stdout [ -- empty -- ] Feb 17 18:42:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:13921 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:42:34 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:42:34 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:42:34 INFO: haproxy daemon running Feb 17 18:42:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13852) = 0: Resource temporarily unavailable (11) Feb 17 18:42:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13877) = 0: Resource temporarily unavailable (11) Feb 17 18:42:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:13950 - exited with rc=0 Feb 17 18:42:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:13950:stderr [ -- empty -- ] Feb 17 18:42:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:13950:stdout [ -- empty -- ] Feb 17 18:42:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:13950 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:42:34 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:34 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:34 INFO: MySQL is not running Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:42:35 [14015] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14015 id=5d019582-afc9-4eda-8407-9b70979cb03a Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14015-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14015] Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14015] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14015] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14015] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14015] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:42:35 [14015] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14015-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14015-11) state:2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14015-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14015-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14015-11-header Feb 17 18:42:35 [14018] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14018 id=b43ec507-6da0-4ba2-8741-6a0cef28cbe4 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14018-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14018] Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14018] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14018] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14018] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:42:35 [14018] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:42:35 [14018] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14018-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14018-11) state:2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14018-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14018-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14018-11-header Feb 17 18:42:35 [14032] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14032 id=3ae13678-b081-4fa3-a300-b15fa6f5540e Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14032-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14032] Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14032] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14032] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14032] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3050 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:42:35 [14032] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3050 for localhost Feb 17 18:42:35 [14032] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14032-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14032-11) state:2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14032-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14032-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14032-11-header Feb 17 18:42:35 [14040] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14040 id=3c3122cf-01e2-4a33-8534-5323d137a122 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14040-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14040] Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14040] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14040] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14040] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:42:35 [14040] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:42:35 [14040] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14040-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14040-11) state:2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14040-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14040-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14040-11-header Feb 17 18:42:35 [14041] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14041 id=d8be0986-f6b9-4532-84e2-43934beb0d38 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14041-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14041] Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14041] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14041] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14041] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:42:35 [14041] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:42:35 [14041] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14041-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14041-11) state:2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14041-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14041-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14041-11-header Feb 17 18:42:35 [14043] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14043 id=14b823f0-02e1-467b-8ca5-1d986307e5a5 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14043-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14043] Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14043] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14043] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14043] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:42:35 [14043] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:42:35 [14043] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14043-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14043-11) state:2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14043-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14043-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14043-11-header Feb 17 18:42:35 [14049] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14049 id=ed538edd-2efb-4ac6-9079-eb9c465b0ec0 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14049-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14049] Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14049] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14049] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14049] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14049] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.66, for localhost Feb 17 18:42:35 [14049] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.66, Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.66,, Current: 0.70,, Stored: 0.70, Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.66, Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.66,) Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:42:35 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/201) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [14055] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 201: cpu_load=0.66, Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201c130 for uid=0 gid=0 pid=14055 id=edca33b6-5719-4ee4-bafa-df9062b8a826 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14055-12) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14055] Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14049-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14049-11) state:2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.613 2 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.614 (null) Feb 17 18:42:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=614 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.66, Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:35 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.614 to 0.155.613 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/201, version=0.155.614) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14049-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14049-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14049-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 201 for cpu_load=0.66, passed Feb 17 18:42:35 [14055] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14055] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14055] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:42:35 [14055] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:42:35 [14055] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14055-12) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14055-12) state:2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14055-12-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14055-12-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14055-12-header Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:35 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/901, version=0.155.614) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:35 [14061] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14061 id=eb881eb6-c10b-44eb-8e69-be44a50c687a Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14061-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14061] Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14061] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14061] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14061] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14061] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=500 for localhost Feb 17 18:42:35 [14061] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=500 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 500, Current: 500, Stored: 500 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14061-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14061-11) state:2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14061-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14061-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14061-11-header Feb 17 18:42:35 [14074] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14074 id=c1a9ede3-0596-47d6-b7a9-2ef715cc342a Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14074-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14074] Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14074] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14074] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14074] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:42:35 [14074] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:42:35 [14074] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14074-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14074-11) state:2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14074-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14074-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14074-11-header Feb 17 18:42:35 [14087] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14087 id=abdc3d87-888f-4626-aed6-458eee921200 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14087-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14087] Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14087] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14087] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14087] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14087] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:42:35 [14087] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14087-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14087-11) state:2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14087-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14087-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14087-11-header Feb 17 18:42:35 [14102] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14102 id=0b50b8a9-fb75-4279-a9db-f37bf02a980e Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14102-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14102] Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14102] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14102] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14102] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9063 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9063, Current: 9063, Stored: 9063 Feb 17 18:42:35 [14102] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9063 for localhost Feb 17 18:42:35 [14102] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14102-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14102-11) state:2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14102-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14102-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14102-11-header Feb 17 18:42:35 [14114] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14114 id=6e9daa56-cc4a-47f7-96f6-592c9e336f41 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14114-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14114] Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14114] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14114] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [14114] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:42:35 [14114] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:42:35 [14114] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13852) = 0: Resource temporarily unavailable (11) Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13877) = 0: Resource temporarily unavailable (11) Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14006 - exited with rc=0 Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14006:stderr [ -- empty -- ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14006:stdout [ arch: x86_64 ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14006:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14006:stdout [ free_swap: 3050 ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14006:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14006:stdout [ cpu_speed: 6999.99 ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14006:stdout [ cpu_cores: 2 ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14006:stdout [ cpu_load: 0.66, ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14006:stdout [ ram_total: 3050 ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14006:stdout [ ram_free: 500 ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14006:stdout [ root_free: 11692 ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14006:stdout [ root_free: 11692 ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14006:stdout [ var_log_free: 9063 ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14006:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:14006 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14114-11) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14114-11) state:2 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14114-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14114-11-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14114-11-header Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df10 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.614 2 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.615 (null) Feb 17 18:42:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=615 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.26 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/903, version=0.155.615) Feb 17 18:42:35 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.615 to 0.155.614 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:42:35 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/203) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df10 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:35 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:35 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/203, version=0.155.615) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:35 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 203 for cpu_load=0.66, passed Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:42:35 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13852) = 0: Resource temporarily unavailable (11) Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13877) = 0: Resource temporarily unavailable (11) Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:14125 - exited with rc=8 Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:14125:stderr [ -- empty -- ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:14125:stdout [ vrouter ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:14125:stdout [ mgmt-conntrd ] Feb 17 18:42:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:14125 exit-code:8 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:42:36 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.671ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.708ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.547ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.524ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:42:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:13852 - exited with rc=0 Feb 17 18:42:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:13852:stderr [ -- empty -- ] Feb 17 18:42:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:13852:stdout [ -- empty -- ] Feb 17 18:42:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:13852 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13877) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:42:36 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.735ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.534ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:42:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:13877 - exited with rc=0 Feb 17 18:42:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:13877:stderr [ -- empty -- ] Feb 17 18:42:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:13877:stdout [ -- empty -- ] Feb 17 18:42:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:13877 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:42:37 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:37 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:37 INFO: MySQL is not running Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:40 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:40 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:40 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:40 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:40 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:40 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:40 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:40 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:40 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 82ac65642959de9e55f61c42f0ce6794 for 0.155.615 (0x1733cf0 0) Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:40 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:40 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:40 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:40 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:40 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:40 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:40 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:40 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-mysql-wss: 2016/02/17_18:42:40 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:40 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:40 INFO: MySQL is not running Feb 17 18:42:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:42:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:42:43 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:43 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:43 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:42:44 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.626ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.652ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.527ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.535ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:42:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:42:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14295 - exited with rc=0 Feb 17 18:42:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14295:stderr [ -- empty -- ] Feb 17 18:42:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14295:stdout [ -- empty -- ] Feb 17 18:42:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:14295 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14314) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:42:44 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.773ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.544ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.554ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:42:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14314 - exited with rc=0 Feb 17 18:42:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14314:stderr [ -- empty -- ] Feb 17 18:42:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14314:stdout [ -- empty -- ] Feb 17 18:42:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:14314 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:42:46 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:46 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:46 INFO: MySQL is not running Feb 17 18:42:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:42:48 INFO: ntp daemon running Feb 17 18:42:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:14483 - exited with rc=0 Feb 17 18:42:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:14483:stderr [ -- empty -- ] Feb 17 18:42:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:14483:stdout [ -- empty -- ] Feb 17 18:42:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:14483 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 ocf-mysql-wss: 2016/02/17_18:42:49 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:49 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:49 INFO: MySQL is not running Feb 17 18:42:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:42:50 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14527) = 0: Resource temporarily unavailable (11) Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14563) = 0: Resource temporarily unavailable (11) Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:14582 - exited with rc=0 Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:14582:stderr [ -- empty -- ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:14582:stdout [ -- empty -- ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:14582 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:42:50 [14617] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14617 id=37a01cce-7254-4d9f-b69b-380f40cc2c0d Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14617-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14617] Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14617] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14617] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14617] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:42:50 [14617] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:42:50 [14617] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14617-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14617-11) state:2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14617-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14617-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14617-11-header Feb 17 18:42:50 [14620] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14620 id=48aefab6-98c6-496e-b58f-30ffaffad364 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14620-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14620] Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14620] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14620] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14620] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:42:50 [14620] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:42:50 [14620] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14620-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14620-11) state:2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14620-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14620-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14620-11-header Feb 17 18:42:50 [14634] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14634 id=b941fc1c-ef55-4950-a179-6a9cbf3f63c8 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14634-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14634] Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14634] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14634] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14634] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3050 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:42:50 [14634] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3050 for localhost Feb 17 18:42:50 [14634] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14634-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14634-11) state:2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14634-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14634-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14634-11-header Feb 17 18:42:50 [14641] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14641 id=9788345b-77da-4281-8d23-56bef7444122 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14641-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14641] Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14641] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14641] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14641] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:42:50 [14641] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:42:50 [14641] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14641-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14641-11) state:2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14641-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14641-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14641-11-header Feb 17 18:42:50 [14642] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14642 id=152f1049-8455-4ba9-bd6c-b156dfbed0cd Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14642-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14642] Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14642] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14642] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14642] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:42:50 [14642] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:42:50 [14642] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14642-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14642-11) state:2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14642-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14642-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14642-11-header Feb 17 18:42:50 [14643] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14643 id=ccc26332-54ad-492c-8d79-570aaee140b0 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14643-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14643] Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14643] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14643] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14643] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:42:50 [14643] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:42:50 [14643] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14643-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14643-11) state:2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14643-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14643-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14643-11-header Feb 17 18:42:50 [14648] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14648 id=9758a84d-17b2-481d-b000-fac797e4b486 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14648-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14648] Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14648] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14648] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14648] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.63, Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.63,, Current: 0.66,, Stored: 0.66, Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.63, Feb 17 18:42:50 [14648] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.63, for localhost Feb 17 18:42:50 [14648] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.63,) Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:42:50 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/205) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 205: cpu_load=0.63, Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14648-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14648-11) state:2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14648-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14648-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14648-11-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df10 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.615 2 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.616 (null) Feb 17 18:42:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=616 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.63, Feb 17 18:42:50 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/205, version=0.155.616) Feb 17 18:42:50 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.616 to 0.155.615 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:50 [14653] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 205 for cpu_load=0.63, passed Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14653 id=8bd8c1f7-e5a9-4008-a1c4-f68aaeae6882 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14653-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14653] Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14653] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14653] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14653] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:42:50 [14653] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:42:50 [14653] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14653-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14653-11) state:2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14653-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14653-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14653-11-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:50 [14658] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14658 id=ad2a1881-83fd-40a2-abd8-42ee575bed02 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14658-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14658] Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df10 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:42:50 [14658] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14658] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14658] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=500 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 500, Current: 500, Stored: 500 Feb 17 18:42:50 [14658] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=500 for localhost Feb 17 18:42:50 [14658] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14658-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14658-11) state:2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14658-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14658-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14658-11-header Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:50 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/905, version=0.155.616) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:50 [14670] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14670 id=229d299e-fb9f-474a-9966-8fe9a26b586f Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14670-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14670] Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14670] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14670] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14670] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:42:50 [14670] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:42:50 [14670] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14670-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14670-11) state:2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14670-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14670-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14670-11-header Feb 17 18:42:50 [14682] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14682 id=82ce80b3-0fd8-4940-95f9-4239c953acf2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14682-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14682] Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14682] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14682] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14682] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:42:50 [14682] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:42:50 [14682] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14682-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14682-11) state:2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14682-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14682-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14682-11-header Feb 17 18:42:50 [14694] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14694 id=df578f87-7a8f-43f5-9822-ceb23bd57f16 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14694-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14694] Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14694] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14694] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14694] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9062 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9062, Current: 9063, Stored: 9063 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9062 Feb 17 18:42:50 [14694] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9062 for localhost Feb 17 18:42:50 [14694] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9062) Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:42:50 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/207) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 207: var_log_free=9062 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.616 2 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.617 (null) Feb 17 18:42:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=617 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9062 Feb 17 18:42:50 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.617 to 0.155.616 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:50 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/207, version=0.155.617) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 207 for var_log_free=9062 passed Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1cf0 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14694-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14694-11) state:2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14694-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14694-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14694-11-header Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:50 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/907, version=0.155.617) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1cf0 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:50 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:50 [14715] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=14715 id=1bc95d54-f6a0-476d-afa3-a379ed71acef Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-14715-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14715] Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14715] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14715] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14715] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:42:50 [14715] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:42:50 [14715] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14527) = 0: Resource temporarily unavailable (11) Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14563) = 0: Resource temporarily unavailable (11) Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14609 - exited with rc=0 Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14609:stderr [ -- empty -- ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14609:stdout [ arch: x86_64 ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14609:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14609:stdout [ free_swap: 3050 ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14609:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14609:stdout [ cpu_speed: 6999.99 ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14609:stdout [ cpu_cores: 2 ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14609:stdout [ cpu_load: 0.63, ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14609:stdout [ ram_total: 3050 ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14609:stdout [ ram_free: 500 ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14609:stdout [ root_free: 11692 ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14609:stdout [ root_free: 11692 ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14609:stdout [ var_log_free: 9062 ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:14609:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:14609 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14695) = 0: Resource temporarily unavailable (11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-14715-11) Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-14715-11) state:2 Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-14715-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-14715-11-header Feb 17 18:42:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-14715-11-header ocf-ns_dns: 2016/02/17_18:42:50 INFO: dnsmasq daemon running Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14527) = 0: Resource temporarily unavailable (11) Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14563) = 0: Resource temporarily unavailable (11) Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:14695 - exited with rc=0 Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:14695:stderr [ -- empty -- ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:14695:stdout [ -- empty -- ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:14695 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14527) = 0: Resource temporarily unavailable (11) Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14563) = 0: Resource temporarily unavailable (11) Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:14726 - exited with rc=0 Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:14726:stderr [ -- empty -- ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:14726:stdout [ -- empty -- ] Feb 17 18:42:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:14726 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:42:52 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.693ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.723ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.536ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.544ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:42:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:42:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14527 - exited with rc=0 Feb 17 18:42:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14527:stderr [ -- empty -- ] Feb 17 18:42:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:14527:stdout [ -- empty -- ] Feb 17 18:42:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:14527 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14563) = 0: Resource temporarily unavailable (11) ocf-mysql-wss: 2016/02/17_18:42:52 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:52 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:52 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:42:52 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.695ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:42:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14563 - exited with rc=0 Feb 17 18:42:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14563:stderr [ -- empty -- ] Feb 17 18:42:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:14563:stdout [ -- empty -- ] Feb 17 18:42:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:14563 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:53 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.000667 (2 ticks in 30s) Feb 17 18:42:53 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.460000 (full: 0.46 0.63 0.40 2/479 14831) Feb 17 18:42:53 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:42:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:42:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:14846 - exited with rc=0 Feb 17 18:42:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:14846:stderr [ -- empty -- ] Feb 17 18:42:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:14846:stdout [ -- empty -- ] Feb 17 18:42:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:14846 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:42:53 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:42:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:14887 - exited with rc=0 Feb 17 18:42:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:14887:stderr [ -- empty -- ] Feb 17 18:42:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:14887:stdout [ -- empty -- ] Feb 17 18:42:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:14887 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:55 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:55 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:55 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:55 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:55 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:55 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:55 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:55 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:42:55 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: af0ccc41e1482b58b67940f933476691 for 0.155.617 (0x1733cf0 0) Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:55 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:55 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:55 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:42:55 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:42:55 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:42:55 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:55 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:42:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-mysql-wss: 2016/02/17_18:42:55 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:55 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:55 INFO: MySQL is not running Feb 17 18:42:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:42:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:42:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:42:58 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:42:58 INFO: net.ipv4.ip_forward = 1 Feb 17 18:42:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:42:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15001) = 0: Resource temporarily unavailable (11) Feb 17 18:42:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15020) = 0: Resource temporarily unavailable (11) Feb 17 18:42:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:15039 - exited with rc=0 Feb 17 18:42:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:15039:stderr [ -- empty -- ] Feb 17 18:42:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:15039:stdout [ -- empty -- ] Feb 17 18:42:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:15039 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:42:58 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:42:58 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:42:58 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:43:00 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.769ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.798ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.543ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.541ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:43:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:43:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:15001 - exited with rc=0 Feb 17 18:43:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:15001:stderr [ -- empty -- ] Feb 17 18:43:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:15001:stdout [ -- empty -- ] Feb 17 18:43:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:15001 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15020) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:43:00 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.816ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.531ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:43:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:15020 - exited with rc=0 Feb 17 18:43:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:15020:stderr [ -- empty -- ] Feb 17 18:43:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:15020:stdout [ -- empty -- ] Feb 17 18:43:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:15020 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:43:01 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:01 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:01 INFO: MySQL is not running Feb 17 18:43:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:43:03 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff Feb 17 18:43:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:15192 - exited with rc=8 Feb 17 18:43:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:15192:stderr [ -- empty -- ] Feb 17 18:43:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:15192:stdout [ vrouter ] Feb 17 18:43:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:15192:stdout [ mgmt-conntrd ] Feb 17 18:43:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:15192 exit-code:8 exec-time:0ms queue-time:0ms Feb 17 18:43:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:43:04 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:43:04 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:43:04 INFO: haproxy daemon running Feb 17 18:43:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:15367 - exited with rc=0 Feb 17 18:43:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:15367:stderr [ -- empty -- ] Feb 17 18:43:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:15367:stdout [ -- empty -- ] Feb 17 18:43:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:15367 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:43:04 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:04 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:04 INFO: MySQL is not running Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:43:05 [15516] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=15516 id=1add9649-9ea4-44c8-bc1b-3529c4ad8f24 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-15516-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15516] Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15516] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15516] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15516] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:43:05 [15516] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:43:05 [15516] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-15516-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-15516-11) state:2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-15516-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-15516-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-15516-11-header Feb 17 18:43:05 [15525] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=15525 id=c21557eb-a5d6-45e7-bf9f-9de9ac12d972 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-15525-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15525] Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15525] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15525] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15525] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:43:05 [15525] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:43:05 [15525] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-15525-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-15525-11) state:2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-15525-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-15525-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-15525-11-header Feb 17 18:43:05 [15552] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=15552 id=2e6ed14c-2917-4fd1-9601-f4828d485d63 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-15552-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15552] Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15552] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15552] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15552] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3050 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:43:05 [15552] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3050 for localhost Feb 17 18:43:05 [15552] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-15552-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-15552-11) state:2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-15552-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-15552-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-15552-11-header Feb 17 18:43:05 [15564] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=15564 id=8943e085-c85a-4842-9f6a-3bdf1a790f04 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-15564-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15564] Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15564] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15564] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15564] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:43:05 [15564] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:43:05 [15564] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-15564-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-15564-11) state:2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-15564-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-15564-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-15564-11-header Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:43:05 [15577] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=15577 id=c4428ddb-7067-435f-aa46-cd1dbcfb7929 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-15577-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15577] Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15577] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15577] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15577] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:43:05 [15577] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:43:05 [15577] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-15577-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-15577-11) state:2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-15577-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-15577-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-15577-11-header Feb 17 18:43:05 [15584] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=15584 id=5a482951-dd82-41c8-ab75-7512b96ca4dc Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-15584-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15584] Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15584] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15584] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15584] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:43:05 [15584] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:43:05 [15584] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-15584-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-15584-11) state:2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-15584-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-15584-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-15584-11-header Feb 17 18:43:05 [15601] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=15601 id=07079522-ddc5-464c-b235-93d1d61d1aee Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-15601-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15601] Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15601] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15601] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15601] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.62, Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.62,, Current: 0.63,, Stored: 0.63, Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.62, Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.62,) Feb 17 18:43:05 [15601] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.62, for localhost Feb 17 18:43:05 [15601] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/209) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 209: cpu_load=0.62, Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-15601-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-15601-11) state:2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-15601-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-15601-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-15601-11-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dd60 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:05 [15610] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dd60 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.617 2 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.618 (null) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=618 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.62, Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/209, version=0.155.618) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:05 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.618 to 0.155.617 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=15610 id=392b4764-606a-459e-91d8-1627a0bd68a5 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-15610-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15610] Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dd60 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 209 for cpu_load=0.62, passed Feb 17 18:43:05 [15610] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15610] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15610] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dd60 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/909, version=0.155.618) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:43:05 [15610] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:43:05 [15610] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-15610-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-15610-11) state:2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-15610-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-15610-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-15610-11-header Feb 17 18:43:05 [15619] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=15619 id=0ed38677-571a-4206-878e-f98bfd606a13 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-15619-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15619] Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15619] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15619] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15619] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=550 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 550, Current: 500, Stored: 500 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of ram_free is 550 Feb 17 18:43:05 [15619] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=550 for localhost Feb 17 18:43:05 [15619] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: ram_free (550) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[19]) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/211) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 211: ram_free=550 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-15619-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-15619-11) state:2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-15619-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-15619-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-15619-11-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dd60 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e2f0 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dd60 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.618 2 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.619 (null) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=619 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-ram_free']: @value=550 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/211, version=0.155.619) Feb 17 18:43:05 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.619 to 0.155.618 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 211 for ram_free=550 passed Feb 17 18:43:05 [15633] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e2f0 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=15633 id=1d70c59f-6ba9-4cbc-8c7d-22b5715b727e Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-15633-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15633] Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15633] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15633] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15633] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:43:05 [15633] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:43:05 [15633] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-15633-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-15633-11) state:2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-15633-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-15633-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-15633-11-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/911, version=0.155.619) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:05 [15645] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=15645 id=4d019973-c7cb-4f31-a7df-446965eced62 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-15645-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15645] Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15645] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15645] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15645] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:43:05 [15645] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:43:05 [15645] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-15645-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-15645-11) state:2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-15645-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-15645-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-15645-11-header Feb 17 18:43:05 [15657] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=15657 id=27ef97be-26e2-4417-b45f-21222c58d7e6 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-15657-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15657] Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15657] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15657] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15657] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9062 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9062, Current: 9062, Stored: 9062 Feb 17 18:43:05 [15657] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9062 for localhost Feb 17 18:43:05 [15657] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-15657-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-15657-11) state:2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-15657-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-15657-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-15657-11-header Feb 17 18:43:05 [15670] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=15670 id=364b679d-20e3-496a-82b1-7783b681e501 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-15670-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15670] Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15670] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15670] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [15670] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:43:05 [15670] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:43:05 [15670] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-15670-11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-15670-11) state:2 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-15670-11-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-15670-11-header Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-15670-11-header Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15503 - exited with rc=0 Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15503:stderr [ -- empty -- ] Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15503:stdout [ arch: x86_64 ] Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15503:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15503:stdout [ free_swap: 3050 ] Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15503:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15503:stdout [ cpu_speed: 6999.99 ] Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15503:stdout [ cpu_cores: 2 ] Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15503:stdout [ cpu_load: 0.62, ] Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15503:stdout [ ram_total: 3050 ] Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15503:stdout [ ram_free: 550 ] Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15503:stdout [ root_free: 11692 ] Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15503:stdout [ root_free: 11692 ] Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15503:stdout [ var_log_free: 9062 ] Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15503:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:15503 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15578) = 0: Resource temporarily unavailable (11) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df10 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.619 2 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.620 (null) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=620 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.27 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/913, version=0.155.620) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:05 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.620 to 0.155.619 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/213) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/213, version=0.155.620) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 213 for cpu_load=0.62, passed Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b10 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.620 2 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.621 (null) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=621 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-ram_free']: @value=600 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.621 to 0.155.620 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/915, version=0.155.621) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[19]) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/215) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b10 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/215, version=0.155.621) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 215 for ram_free=550 passed Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dd60 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.621 2 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.622 (null) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=622 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-root_free']: @value=11691 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/917, version=0.155.622) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:43:05 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.622 to 0.155.621 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/217) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dd60 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/217, version=0.155.622) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 217 for root_free=11692 passed Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/919, version=0.155.622) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/219) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/219, version=0.155.622) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 219 for root_free=11692 passed Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dfc0 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.622 2 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.623 (null) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=623 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9061 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/921, version=0.155.623) Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:43:05 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.623 to 0.155.622 Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/221) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dfc0 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:05 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/221, version=0.155.623) Feb 17 18:43:05 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 221 for var_log_free=9062 passed Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:05 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-mysql-wss: 2016/02/17_18:43:07 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:07 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:07 INFO: MySQL is not running Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-metadata-agent_monitor_60000 Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15578) = 0: Resource temporarily unavailable (11) Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15671) = 0: Resource temporarily unavailable (11) Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:15742 - exited with rc=0 Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:15742:stderr [ -- empty -- ] Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:15742:stdout [ -- empty -- ] Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-metadata-agent action:monitor call_id:131 pid:15742 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:43:08 INFO: ntp daemon running Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15578) = 0: Resource temporarily unavailable (11) Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15671) = 0: Resource temporarily unavailable (11) Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:15767 - exited with rc=0 Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:15767:stderr [ -- empty -- ] Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:15767:stdout [ -- empty -- ] Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:15767 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:43:08 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.682ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.871ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.545ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.543ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:15578 - exited with rc=0 Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:15578:stderr [ -- empty -- ] Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:15578:stdout [ -- empty -- ] Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:15578 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15671) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:43:08 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.793ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.543ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.544ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:15671 - exited with rc=0 Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:15671:stderr [ -- empty -- ] Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:15671:stdout [ -- empty -- ] Feb 17 18:43:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:15671 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:43:10 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:15825 - exited with rc=0 Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:15825:stderr [ -- empty -- ] Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:15825:stdout [ -- empty -- ] Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:15825 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 ocf-ns_dns: 2016/02/17_18:43:10 INFO: dnsmasq daemon running Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:15852 - exited with rc=0 Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:15852:stderr [ -- empty -- ] Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:15852:stdout [ -- empty -- ] Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:15852 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:15865 - exited with rc=0 Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:15865:stderr [ -- empty -- ] Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:15865:stdout [ -- empty -- ] Feb 17 18:43:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:15865 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:43:10 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:10 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:10 INFO: MySQL is not running Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:10 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:10 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:10 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 759a457e72797949ad7d3f8409f80d01 for 0.155.623 (0x1733cf0 0) Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:10 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 ocf-mysql-wss: 2016/02/17_18:43:13 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:13 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:13 INFO: MySQL is not running Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15973) = 0: Resource temporarily unavailable (11) Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:16009 - exited with rc=0 Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:16009:stderr [ -- empty -- ] Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:16009:stdout [ -- empty -- ] Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:16009 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:43:13 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(15973) = 0: Resource temporarily unavailable (11) Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:16034 - exited with rc=0 Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:16034:stderr [ -- empty -- ] Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:16034:stdout [ -- empty -- ] Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:16034 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:43:16 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.646ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.674ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.544ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.542ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:43:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:43:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:15973 - exited with rc=0 Feb 17 18:43:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:15973:stderr [ -- empty -- ] Feb 17 18:43:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:15973:stdout [ -- empty -- ] Feb 17 18:43:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:15973 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(16063) = 0: Resource temporarily unavailable (11) ocf-mysql-wss: 2016/02/17_18:43:16 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:16 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:16 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:43:16 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.903ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.532ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.528ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:43:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:16063 - exited with rc=0 Feb 17 18:43:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:16063:stderr [ -- empty -- ] Feb 17 18:43:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:16063:stdout [ -- empty -- ] Feb 17 18:43:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:16063 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:43:19 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:19 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:19 INFO: MySQL is not running Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:43:20 [16243] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=16243 id=c6752b2d-e81b-461e-8efd-0ce5b79fd046 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-16243-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16243] Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16243] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16243] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16243] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:43:20 [16243] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:43:20 [16243] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-16243-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-16243-11) state:2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-16243-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-16243-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-16243-11-header Feb 17 18:43:20 [16246] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=16246 id=2cf62b34-d31d-4043-ab6e-97293c413ea3 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-16246-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16246] Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16246] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16246] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16246] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:43:20 [16246] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:43:20 [16246] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-16246-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-16246-11) state:2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-16246-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-16246-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-16246-11-header Feb 17 18:43:20 [16260] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=16260 id=c8933945-7204-4c29-87a1-b661ef2c725b Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-16260-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16260] Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16260] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16260] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16260] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3050 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:43:20 [16260] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3050 for localhost Feb 17 18:43:20 [16260] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-16260-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-16260-11) state:2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-16260-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-16260-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-16260-11-header Feb 17 18:43:20 [16267] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=16267 id=3f8acb94-80c1-47ed-9c5f-966ea261ad88 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-16267-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16267] Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16267] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16267] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16267] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:43:20 [16267] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:43:20 [16267] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-16267-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-16267-11) state:2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-16267-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-16267-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-16267-11-header Feb 17 18:43:20 [16268] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=16268 id=fbf30881-4283-48ec-815e-dc49416b93df Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-16268-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16268] Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16268] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16268] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16268] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:43:20 [16268] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:43:20 [16268] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-16268-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-16268-11) state:2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-16268-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-16268-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-16268-11-header Feb 17 18:43:20 [16269] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=16269 id=16f06de7-4ff9-4ead-b3c8-b2dfdd4ea70b Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-16269-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16269] Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16269] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16269] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16269] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:43:20 [16269] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:43:20 [16269] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-16269-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-16269-11) state:2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-16269-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-16269-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-16269-11-header Feb 17 18:43:20 [16274] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=16274 id=b8fce6bc-4a7a-495d-993f-d3ee9e26e255 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-16274-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16274] Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16274] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16274] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16274] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.59, Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.59,, Current: 0.62,, Stored: 0.62, Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.59, Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.59,) Feb 17 18:43:20 [16274] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.59, for localhost Feb 17 18:43:20 [16274] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 223: cpu_load=0.59, Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-16274-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-16274-11) state:2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-16274-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-16274-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-16274-11-header Feb 17 18:43:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/223) Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.623 2 Feb 17 18:43:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.624 (null) Feb 17 18:43:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=624 Feb 17 18:43:20 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.59, Feb 17 18:43:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/223, version=0.155.624) Feb 17 18:43:20 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.624 to 0.155.623 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:20 [16283] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 223 for cpu_load=0.59, passed Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=16283 id=b3965c1e-18c2-4175-9ccb-857248a6b137 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-16283-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16283] Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:20 [16283] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:20 [16283] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:20 [16283] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:43:20 [16283] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:43:20 [16283] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-16283-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-16283-11) state:2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-16283-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-16283-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-16283-11-header Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:20 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:20 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/923, version=0.155.624) Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:20 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:20 [16288] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=16288 id=e26f1d6d-a29b-42e0-be1d-24255553f043 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-16288-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16288] Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16288] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16288] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16288] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=550 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 550, Current: 550, Stored: 550 Feb 17 18:43:20 [16288] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=550 for localhost Feb 17 18:43:20 [16288] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-16288-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-16288-11) state:2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-16288-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-16288-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-16288-11-header Feb 17 18:43:20 [16300] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=16300 id=d07dbd9d-21fb-4295-9345-bad6eb1b12fc Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-16300-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16300] Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16300] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16300] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16300] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:43:20 [16300] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:43:20 [16300] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-16300-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-16300-11) state:2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-16300-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-16300-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-16300-11-header Feb 17 18:43:20 [16312] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=16312 id=87ac6787-444a-4e6f-991e-6e88b20b6d0b Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-16312-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16312] Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16312] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16312] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16312] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:43:20 [16312] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:43:20 [16312] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-16312-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-16312-11) state:2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-16312-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-16312-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-16312-11-header Feb 17 18:43:20 [16324] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=16324 id=3fe2e80e-39fe-4217-a78d-eb08a3b19a94 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-16324-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16324] Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16324] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16324] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16324] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9062 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9062, Current: 9062, Stored: 9062 Feb 17 18:43:20 [16324] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9062 for localhost Feb 17 18:43:20 [16324] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-16324-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-16324-11) state:2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-16324-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-16324-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-16324-11-header Feb 17 18:43:20 [16336] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=16336 id=a7cfdac7-8501-4a6c-89cf-26d129b06b26 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-16336-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16336] Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16336] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16336] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [16336] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:43:20 [16336] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:43:20 [16336] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-16336-11) Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-16336-11) state:2 Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-16336-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-16336-11-header Feb 17 18:43:20 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-16336-11-header Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16235 - exited with rc=0 Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16235:stderr [ -- empty -- ] Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16235:stdout [ arch: x86_64 ] Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16235:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16235:stdout [ free_swap: 3050 ] Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16235:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16235:stdout [ cpu_speed: 6999.99 ] Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16235:stdout [ cpu_cores: 2 ] Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16235:stdout [ cpu_load: 0.59, ] Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16235:stdout [ ram_total: 3050 ] Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16235:stdout [ ram_free: 550 ] Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16235:stdout [ root_free: 11692 ] Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16235:stdout [ root_free: 11692 ] Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16235:stdout [ var_log_free: 9062 ] Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16235:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:43:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:16235 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:43:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:43:22 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:22 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:22 INFO: MySQL is not running Feb 17 18:43:23 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.001000 (3 ticks in 30s) Feb 17 18:43:23 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.340000 (full: 0.34 0.59 0.40 2/481 16439) Feb 17 18:43:23 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 ocf-ns_IPaddr2: 2016/02/17_18:43:24 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.641ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.668ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.533ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.558ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:43:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:43:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:16366 - exited with rc=0 Feb 17 18:43:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:16366:stderr [ -- empty -- ] Feb 17 18:43:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:16366:stdout [ -- empty -- ] Feb 17 18:43:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:16366 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(16385) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:43:25 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.777ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.529ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:43:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:16385 - exited with rc=0 Feb 17 18:43:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:16385:stderr [ -- empty -- ] Feb 17 18:43:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:16385:stdout [ -- empty -- ] Feb 17 18:43:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:16385 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:43:25 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:25 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:25 INFO: MySQL is not running Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:25 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 88170c18e05f46d6c8c00f2753018970 for 0.155.624 (0x1733cf0 0) Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:25 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:25 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:43:28 INFO: ntp daemon running Feb 17 18:43:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:16553 - exited with rc=0 Feb 17 18:43:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:16553:stderr [ -- empty -- ] Feb 17 18:43:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:16553:stdout [ -- empty -- ] Feb 17 18:43:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:16553 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:43:28 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:43:28 INFO: net.ipv4.ip_forward = 1 Feb 17 18:43:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:16564 - exited with rc=0 Feb 17 18:43:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:16564:stderr [ -- empty -- ] Feb 17 18:43:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:16564:stdout [ -- empty -- ] Feb 17 18:43:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:16564 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:43:28 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:28 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:28 INFO: MySQL is not running Feb 17 18:43:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:43:30 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(16633) = 0: Resource temporarily unavailable (11) Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(16652) = 0: Resource temporarily unavailable (11) Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:16671 - exited with rc=8 Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:16671:stderr [ -- empty -- ] Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:16671:stdout [ vrouter ] Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:16671:stdout [ mgmt-conntrd ] Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:16671 exit-code:8 exec-time:0ms queue-time:0ms Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:43:30 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(16633) = 0: Resource temporarily unavailable (11) Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(16652) = 0: Resource temporarily unavailable (11) Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:16692 - exited with rc=0 Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:16692:stderr [ -- empty -- ] Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:16692:stdout [ -- empty -- ] Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:16692 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 ocf-ns_dns: 2016/02/17_18:43:30 INFO: dnsmasq daemon running Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(16633) = 0: Resource temporarily unavailable (11) Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(16652) = 0: Resource temporarily unavailable (11) Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:16719 - exited with rc=0 Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:16719:stderr [ -- empty -- ] Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:16719:stdout [ -- empty -- ] Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:16719 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(16633) = 0: Resource temporarily unavailable (11) Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(16652) = 0: Resource temporarily unavailable (11) Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:16732 - exited with rc=0 Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:16732:stderr [ -- empty -- ] Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:16732:stdout [ -- empty -- ] Feb 17 18:43:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:16732 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:43:31 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:31 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:31 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:43:32 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.671ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.702ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.559ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.542ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:43:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:43:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:16633 - exited with rc=0 Feb 17 18:43:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:16633:stderr [ -- empty -- ] Feb 17 18:43:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:16633:stdout [ -- empty -- ] Feb 17 18:43:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:16633 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(16652) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:43:33 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.667ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.528ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.526ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:16652 - exited with rc=0 Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:16652:stderr [ -- empty -- ] Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:16652:stdout [ -- empty -- ] Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:16652 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:16854 - exited with rc=0 Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:16854:stderr [ -- empty -- ] Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:16854:stdout [ -- empty -- ] Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:16854 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:43:33 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:16879 - exited with rc=0 Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:16879:stderr [ -- empty -- ] Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:16879:stdout [ -- empty -- ] Feb 17 18:43:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:16879 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:43:34 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:43:34 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:43:34 INFO: haproxy daemon running Feb 17 18:43:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:16908 - exited with rc=0 Feb 17 18:43:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:16908:stderr [ -- empty -- ] Feb 17 18:43:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:16908:stdout [ -- empty -- ] Feb 17 18:43:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:16908 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:43:34 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:34 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:34 INFO: MySQL is not running Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:43:35 [16992] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=16992 id=75159344-c708-4542-904d-193b23fd31c1 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-16992-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16992] Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [16992] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [16992] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [16992] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:43:35 [16992] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:43:35 [16992] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-16992-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-16992-11) state:2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-16992-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-16992-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-16992-11-header Feb 17 18:43:35 [16995] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=16995 id=5f5cd311-dcea-4758-bb6b-f66f030aa4af Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-16995-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16995] Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [16995] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [16995] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [16995] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:43:35 [16995] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:43:35 [16995] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-16995-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-16995-11) state:2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-16995-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-16995-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-16995-11-header Feb 17 18:43:35 [17010] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17010 id=ec14c570-fdca-4188-915d-5d4862bb204b Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17010-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17010] Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17010] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17010] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17010] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3050 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:43:35 [17010] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3050 for localhost Feb 17 18:43:35 [17010] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17010-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17010-11) state:2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17010-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17010-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17010-11-header Feb 17 18:43:35 [17017] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17017 id=b156219f-9e5e-466f-870a-61abb9e97abf Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17017-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17017] Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17017] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17017] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17017] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:43:35 [17017] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:43:35 [17017] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17017-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17017-11) state:2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17017-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17017-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17017-11-header Feb 17 18:43:35 [17018] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17018 id=cc626307-afd5-4e09-86e2-efa2b83233c0 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17018-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17018] Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17018] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17018] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17018] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:43:35 [17018] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:43:35 [17018] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17018-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17018-11) state:2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17018-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17018-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17018-11-header Feb 17 18:43:35 [17019] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17019 id=c598f4bb-9ab8-4787-b770-0544d6aac168 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17019-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17019] Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17019] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17019] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17019] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:43:35 [17019] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:43:35 [17019] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17019-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17019-11) state:2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17019-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17019-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17019-11-header Feb 17 18:43:35 [17024] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17024 id=29bb3788-ab1b-4b0c-8d3c-986cf276b9c7 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17024-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17024] Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17024] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17024] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17024] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.56, Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.56,, Current: 0.59,, Stored: 0.59, Feb 17 18:43:35 [17024] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.56, for localhost Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.56, Feb 17 18:43:35 [17024] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.56,) Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:43:35 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/225) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 225: cpu_load=0.56, Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17024-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17024-11) state:2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17024-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17024-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17024-11-header Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dd60 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dd60 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.624 2 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.625 (null) Feb 17 18:43:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=625 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.56, Feb 17 18:43:35 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/225, version=0.155.625) Feb 17 18:43:35 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.625 to 0.155.624 Feb 17 18:43:35 [17030] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 225 for cpu_load=0.56, passed Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17030 id=2092bfc6-6d2a-439e-92db-64ae404ae350 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17030-11) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17030] Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:35 [17030] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17030] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17030] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:43:35 [17030] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:43:35 [17030] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17030-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17030-11) state:2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17030-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17030-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17030-11-header Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:35 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/925, version=0.155.625) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:35 [17035] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17035 id=70dde830-b0b1-44e3-b33b-83663cd4a512 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17035-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17035] Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17035] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17035] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17035] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=550 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 550, Current: 550, Stored: 550 Feb 17 18:43:35 [17035] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=550 for localhost Feb 17 18:43:35 [17035] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17035-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17035-11) state:2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17035-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17035-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17035-11-header Feb 17 18:43:35 [17047] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17047 id=76188542-f1a0-423d-a20b-cee0c35110da Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17047-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17047] Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17047] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17047] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17047] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:43:35 [17047] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:43:35 [17047] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17047-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17047-11) state:2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17047-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17047-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17047-11-header Feb 17 18:43:35 [17059] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17059 id=7305a343-e07d-4062-8e3c-835aee40431a Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17059-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17059] Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17059] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17059] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17059] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:43:35 [17059] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:43:35 [17059] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17059-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17059-11) state:2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17059-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17059-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17059-11-header Feb 17 18:43:35 [17071] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17071 id=b22d3a77-e052-4328-af8d-ac2e5eaa908a Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17071-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17071] Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17071] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17071] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [17071] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9061 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9061, Current: 9062, Stored: 9062 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9061 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9061) Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:43:35 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/227) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 227: var_log_free=9061 Feb 17 18:43:35 [17071] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9061 for localhost Feb 17 18:43:35 [17071] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1cd0 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.625 2 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.626 (null) Feb 17 18:43:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=626 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9061 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/227, version=0.155.626) Feb 17 18:43:35 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.626 to 0.155.625 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 227 for var_log_free=9061 passed Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17071-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17071-11) state:2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17071-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17071-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17071-11-header Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1cd0 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [17083] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17083 id=98b381b3-124f-4e57-be62-4a73bc1dc26c Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17083-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17083] Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:35 [17083] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:35 [17083] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:35 [17083] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:43:35 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/927, version=0.155.626) Feb 17 18:43:35 [17083] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:43:35 [17083] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:35 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17083-11) Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17083-11) state:2 Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17083-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17083-11-header Feb 17 18:43:35 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17083-11-header Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16984 - exited with rc=0 Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16984:stderr [ -- empty -- ] Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16984:stdout [ arch: x86_64 ] Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16984:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16984:stdout [ free_swap: 3050 ] Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16984:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16984:stdout [ cpu_speed: 6999.99 ] Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16984:stdout [ cpu_cores: 2 ] Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16984:stdout [ cpu_load: 0.56, ] Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16984:stdout [ ram_total: 3050 ] Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16984:stdout [ ram_free: 550 ] Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16984:stdout [ root_free: 11692 ] Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16984:stdout [ root_free: 11692 ] Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16984:stdout [ var_log_free: 9061 ] Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16984:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:43:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:16984 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:36 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.626 2 Feb 17 18:43:36 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.627 (null) Feb 17 18:43:36 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=627 Feb 17 18:43:36 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.29 Feb 17 18:43:36 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/929, version=0.155.627) Feb 17 18:43:36 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.627 to 0.155.626 Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:36 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:43:36 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/229) Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:36 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:36 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/229, version=0.155.627) Feb 17 18:43:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:36 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 229 for cpu_load=0.56, passed Feb 17 18:43:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 ocf-mysql-wss: 2016/02/17_18:43:37 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:37 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:37 INFO: MySQL is not running Feb 17 18:43:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:43:40 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.651ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.685ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.536ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.539ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:43:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:43:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17119 - exited with rc=0 Feb 17 18:43:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17119:stderr [ -- empty -- ] Feb 17 18:43:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17119:stdout [ -- empty -- ] Feb 17 18:43:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:17119 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17155) = 0: Resource temporarily unavailable (11) ocf-mysql-wss: 2016/02/17_18:43:40 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:40 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:40 INFO: MySQL is not running Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:41 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:41 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:41 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:41 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:41 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:41 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:41 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:41 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:41 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 137474113b1fe8f2dfd38d43c130112c for 0.155.627 (0x1733cf0 0) Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:41 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:41 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:41 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:41 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:41 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:41 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:41 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_IPaddr2: 2016/02/17_18:43:41 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.765ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.537ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.537ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:43:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17155 - exited with rc=0 Feb 17 18:43:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17155:stderr [ -- empty -- ] Feb 17 18:43:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17155:stdout [ -- empty -- ] Feb 17 18:43:41 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:17155 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:43:43 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:43 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:43 INFO: MySQL is not running Feb 17 18:43:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:43:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:43:46 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:46 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:46 INFO: MySQL is not running Feb 17 18:43:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:43:48 INFO: ntp daemon running Feb 17 18:43:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17354) = 0: Resource temporarily unavailable (11) Feb 17 18:43:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17373) = 0: Resource temporarily unavailable (11) Feb 17 18:43:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:17444 - exited with rc=0 Feb 17 18:43:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:17444:stderr [ -- empty -- ] Feb 17 18:43:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:17444:stdout [ -- empty -- ] Feb 17 18:43:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:17444 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:43:48 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.666ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.687ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.541ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.540ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:43:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17354 - exited with rc=0 Feb 17 18:43:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17354:stderr [ -- empty -- ] Feb 17 18:43:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17354:stdout [ -- empty -- ] Feb 17 18:43:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:17354 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17373) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:43:49 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.913ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.536ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.538ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:43:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17373 - exited with rc=0 Feb 17 18:43:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17373:stderr [ -- empty -- ] Feb 17 18:43:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17373:stdout [ -- empty -- ] Feb 17 18:43:49 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:17373 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:43:49 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:49 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:49 INFO: MySQL is not running Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:43:50 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:17520 - exited with rc=0 Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:17520:stderr [ -- empty -- ] Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:17520:stdout [ -- empty -- ] Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:17520 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 ocf-ns_dns: 2016/02/17_18:43:50 INFO: dnsmasq daemon running Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:17547 - exited with rc=0 Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:17547:stderr [ -- empty -- ] Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:17547:stdout [ -- empty -- ] Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:17547 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:17560 - exited with rc=0 Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:17560:stderr [ -- empty -- ] Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:17560:stdout [ -- empty -- ] Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:17560 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:43:50 [17598] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17598 id=c3cdccdf-d26c-4809-a02a-ecbb8e811800 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17598-11) Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17598] Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [17598] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [17598] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [17598] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:43:50 [17598] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:43:50 [17598] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17598-11) Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17598-11) state:2 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17598-11-header Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17598-11-header Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17598-11-header Feb 17 18:43:50 [17601] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17601 id=50bb4d2b-68d8-44e7-bf29-a91e28209cba Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17601-11) Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17601] Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [17601] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [17601] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [17601] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:43:50 [17601] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:43:50 [17601] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17601-11) Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17601-11) state:2 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17601-11-header Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17601-11-header Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17601-11-header Feb 17 18:43:50 [17615] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17615 id=3085174f-ef1a-4b18-8faf-2f8a11d1e1c9 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17615-11) Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17615] Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [17615] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [17615] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [17615] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3050 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:43:50 [17615] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3050 for localhost Feb 17 18:43:50 [17615] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17615-11) Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17615-11) state:2 Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17615-11-header Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17615-11-header Feb 17 18:43:50 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17615-11-header Feb 17 18:43:50 [17622] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17622 id=7b108024-b1b5-49ea-a8c1-23c2b6b70313 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17622-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17622] Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17622] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17622] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17622] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:43:51 [17622] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:43:51 [17622] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17622-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17622-11) state:2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17622-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17622-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17622-11-header Feb 17 18:43:51 [17623] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17623 id=7582f844-8b28-4582-81dc-7ec0c48eb3a7 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17623-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17623] Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17623] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17623] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17623] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:43:51 [17623] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:43:51 [17623] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17623-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17623-11) state:2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17623-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17623-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17623-11-header Feb 17 18:43:51 [17624] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17624 id=e293ce8f-9378-43e9-be78-46a575c6db21 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17624-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17624] Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17624] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17624] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17624] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:43:51 [17624] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:43:51 [17624] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17624-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17624-11) state:2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17624-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17624-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17624-11-header Feb 17 18:43:51 [17629] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17629 id=22a69c6c-0681-4ccf-bd9e-49d1b82303f7 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17629-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17629] Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17629] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17629] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17629] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.53, Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.53,, Current: 0.56,, Stored: 0.56, Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.53, Feb 17 18:43:51 [17629] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.53, for localhost Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.53,) Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:43:51 [17629] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:43:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/231) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 231: cpu_load=0.53, Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.627 2 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.628 (null) Feb 17 18:43:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=628 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.53, Feb 17 18:43:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/231, version=0.155.628) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 231 for cpu_load=0.53, passed Feb 17 18:43:51 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.628 to 0.155.627 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17629-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17629-11) state:2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17629-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17629-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17629-11-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [17635] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17635 id=048af058-f083-4a3a-8ebd-ef781050bdf0 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17635-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17635] Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17635] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17635] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17635] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:43:51 [17635] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:43:51 [17635] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17635-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17635-11) state:2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17635-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17635-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17635-11-header Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dfa0 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:51 [17641] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/931, version=0.155.628) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dfa0 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17641 id=da76040e-716a-40eb-a1f4-621adf83c0dd Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17641-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17641] Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17641] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17641] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17641] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=550 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 550, Current: 550, Stored: 550 Feb 17 18:43:51 [17641] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=550 for localhost Feb 17 18:43:51 [17641] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17641-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17641-11) state:2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17641-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17641-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17641-11-header Feb 17 18:43:51 [17653] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17653 id=99b975f8-a0a9-42f4-8218-f309ff956db8 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17653-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17653] Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17653] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17653] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17653] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:43:51 [17653] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:43:51 [17653] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17653-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17653-11) state:2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17653-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17653-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17653-11-header Feb 17 18:43:51 [17666] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17666 id=7a8646f4-003f-428a-871c-55e84b7198f8 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17666-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17666] Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17666] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17666] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17666] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:43:51 [17666] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:43:51 [17666] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17666-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17666-11) state:2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17666-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17666-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17666-11-header Feb 17 18:43:51 [17679] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17679 id=589b057c-71b4-4415-beb2-4ce16c654065 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17679-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17679] Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17679] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17679] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17679] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9061 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9061, Current: 9061, Stored: 9061 Feb 17 18:43:51 [17679] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9061 for localhost Feb 17 18:43:51 [17679] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17679-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17679-11) state:2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17679-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17679-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17679-11-header Feb 17 18:43:51 [17691] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=17691 id=3821d849-111f-479f-893a-16c0260db6c6 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-17691-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17691] Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17691] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17691] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17691] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:43:51 [17691] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:43:51 [17691] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:17590 - exited with rc=0 Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:17590:stderr [ -- empty -- ] Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:17590:stdout [ arch: x86_64 ] Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:17590:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:17590:stdout [ free_swap: 3050 ] Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:17590:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:17590:stdout [ cpu_speed: 6999.99 ] Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:17590:stdout [ cpu_cores: 2 ] Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:17590:stdout [ cpu_load: 0.53, ] Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:17590:stdout [ ram_total: 3050 ] Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:17590:stdout [ ram_free: 550 ] Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:17590:stdout [ root_free: 11692 ] Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:17590:stdout [ root_free: 11692 ] Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:17590:stdout [ var_log_free: 9061 ] Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:17590:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:43:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:17590 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-17691-11) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-17691-11) state:2 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-17691-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-17691-11-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-17691-11-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df10 Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.628 2 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.629 (null) Feb 17 18:43:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=629 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.28 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/933, version=0.155.629) Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:51 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.629 to 0.155.628 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:43:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/233) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df10 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/233, version=0.155.629) Feb 17 18:43:51 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 233 for cpu_load=0.53, passed ocf-mysql-wss: 2016/02/17_18:43:52 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:52 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:52 INFO: MySQL is not running Feb 17 18:43:53 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.000667 (2 ticks in 30s) Feb 17 18:43:53 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.210000 (full: 0.21 0.53 0.39 2/476 17746) Feb 17 18:43:53 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:43:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:43:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:17784 - exited with rc=0 Feb 17 18:43:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:17784:stderr [ -- empty -- ] Feb 17 18:43:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:17784:stdout [ -- empty -- ] Feb 17 18:43:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:17784 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:43:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:43:53 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:43:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17809) = 0: Resource temporarily unavailable (11) Feb 17 18:43:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:17828 - exited with rc=0 Feb 17 18:43:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:17828:stderr [ -- empty -- ] Feb 17 18:43:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:17828:stdout [ -- empty -- ] Feb 17 18:43:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:17828 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:43:55 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:55 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:55 INFO: MySQL is not running Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:56 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:56 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:56 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:56 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:56 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:56 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:56 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:56 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:43:56 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: e249facada96201d3f9cf68dec352073 for 0.155.629 (0x1733cf0 0) Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:56 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:56 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:56 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:43:56 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:43:56 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:43:56 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:56 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:43:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_IPaddr2: 2016/02/17_18:43:56 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.635ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.653ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.537ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.528ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:43:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17809 - exited with rc=0 Feb 17 18:43:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17809:stderr [ -- empty -- ] Feb 17 18:43:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:17809:stdout [ -- empty -- ] Feb 17 18:43:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:17809 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17857) = 0: Resource temporarily unavailable (11) Feb 17 18:43:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:43:57 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff ocf-ns_IPaddr2: 2016/02/17_18:43:57 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.798ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.533ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:43:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17857 - exited with rc=0 Feb 17 18:43:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17857:stderr [ -- empty -- ] Feb 17 18:43:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:17857:stdout [ -- empty -- ] Feb 17 18:43:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:17857 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:43:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17954) = 0: Resource temporarily unavailable (11) Feb 17 18:43:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:17954 - exited with rc=8 Feb 17 18:43:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:17954:stderr [ -- empty -- ] Feb 17 18:43:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:17954:stdout [ vrouter ] Feb 17 18:43:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:17954:stdout [ mgmt-conntrd ] Feb 17 18:43:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:17954 exit-code:8 exec-time:0ms queue-time:0ms Feb 17 18:43:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:43:58 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:43:58 INFO: net.ipv4.ip_forward = 1 Feb 17 18:43:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:43:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:17997 - exited with rc=0 Feb 17 18:43:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:17997:stderr [ -- empty -- ] Feb 17 18:43:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:17997:stdout [ -- empty -- ] Feb 17 18:43:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:17997 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:43:58 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:43:58 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:43:58 INFO: MySQL is not running ocf-mysql-wss: 2016/02/17_18:44:01 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:01 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:01 INFO: MySQL is not running Feb 17 18:44:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:44:02 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:44:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:44:04 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:44:04 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:44:04 INFO: haproxy daemon running Feb 17 18:44:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(18127) = 0: Resource temporarily unavailable (11) Feb 17 18:44:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(18146) = 0: Resource temporarily unavailable (11) Feb 17 18:44:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:18331 - exited with rc=0 Feb 17 18:44:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:18331:stderr [ -- empty -- ] Feb 17 18:44:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:18331:stdout [ -- empty -- ] Feb 17 18:44:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:18331 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:44:04 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:04 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:04 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:44:04 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.674ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.700ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.536ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.548ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:44:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:18127 - exited with rc=0 Feb 17 18:44:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:18127:stderr [ -- empty -- ] Feb 17 18:44:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:18127:stdout [ -- empty -- ] Feb 17 18:44:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:18127 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(18146) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:44:05 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.671ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.531ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.543ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:44:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:18146 - exited with rc=0 Feb 17 18:44:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:18146:stderr [ -- empty -- ] Feb 17 18:44:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:18146:stdout [ -- empty -- ] Feb 17 18:44:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:18146 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:44:06 [18543] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=18543 id=f2a9270a-1fb7-413e-b184-e836a88f74f6 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-18543-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18543] Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18543] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18543] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18543] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:44:06 [18543] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:44:06 [18543] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-18543-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-18543-11) state:2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-18543-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-18543-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-18543-11-header Feb 17 18:44:06 [18547] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=18547 id=cd3177e8-a27a-48d2-9eab-2f6f2b8d9365 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-18547-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18547] Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18547] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18547] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18547] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:44:06 [18547] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:44:06 [18547] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-18547-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-18547-11) state:2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-18547-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-18547-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-18547-11-header Feb 17 18:44:06 [18561] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=18561 id=9527f384-eef2-44e2-ba00-65473bde200e Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-18561-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18561] Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18561] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18561] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18561] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3050 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:44:06 [18561] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3050 for localhost Feb 17 18:44:06 [18561] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-18561-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-18561-11) state:2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-18561-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-18561-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-18561-11-header Feb 17 18:44:06 [18568] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=18568 id=b4f8e94e-2c8a-4fa9-bc84-fc88e8a2b3fc Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-18568-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18568] Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18568] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18568] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18568] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:44:06 [18568] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:44:06 [18568] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-18568-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-18568-11) state:2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-18568-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-18568-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-18568-11-header Feb 17 18:44:06 [18569] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=18569 id=be1bb29c-c308-4d50-b25a-3ef0523bfe6f Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-18569-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18569] Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18569] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18569] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18569] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:44:06 [18569] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:44:06 [18569] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-18569-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-18569-11) state:2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-18569-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-18569-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-18569-11-header Feb 17 18:44:06 [18570] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=18570 id=38867221-fd37-40d7-90ef-176d215674f5 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-18570-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18570] Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18570] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18570] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18570] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:44:06 [18570] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:44:06 [18570] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-18570-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-18570-11) state:2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-18570-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-18570-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-18570-11-header Feb 17 18:44:06 [18575] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=18575 id=2f4703b9-32b0-494c-b14d-33967b75e634 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-18575-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18575] Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18575] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18575] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18575] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.57, Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.57,, Current: 0.53,, Stored: 0.53, Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.57, Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.57,) Feb 17 18:44:06 [18575] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.57, for localhost Feb 17 18:44:06 [18575] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 235: cpu_load=0.57, Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/235) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-18575-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-18575-11) state:2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-18575-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-18575-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-18575-11-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df20 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.629 2 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.630 (null) Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=630 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.57, Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/235, version=0.155.630) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.630 to 0.155.629 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 235 for cpu_load=0.57, passed Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df20 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:06 [18580] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/935, version=0.155.630) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=18580 id=e438e615-3476-4c9b-918d-c1610e2ccd9f Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-18580-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18580] Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18580] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18580] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18580] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:44:06 [18580] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:44:06 [18580] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-18580-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-18580-11) state:2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-18580-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-18580-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-18580-11-header Feb 17 18:44:06 [18585] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=18585 id=771f67d6-9506-4b21-83ff-64cead5b8c01 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-18585-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18585] Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18585] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18585] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18585] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=600 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 600, Current: 550, Stored: 550 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of ram_free is 600 Feb 17 18:44:06 [18585] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=600 for localhost Feb 17 18:44:06 [18585] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: ram_free (600) Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[19]) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 237: ram_free=600 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/237) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-18585-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-18585-11) state:2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-18585-11-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-18585-11-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-18585-11-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1d60 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.630 2 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.631 (null) Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=631 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-ram_free']: @value=600 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/237, version=0.155.631) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.631 to 0.155.630 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1d60 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 237 for ram_free=600 passed Feb 17 18:44:06 [18597] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=18597 id=04aabd4d-6bfa-4847-adb0-ff709c73ae8b Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-18597-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18597] Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18597] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18597] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18597] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:44:06 [18597] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:44:06 [18597] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-18597-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-18597-11) state:2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-18597-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-18597-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-18597-11-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/937, version=0.155.631) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [18609] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=18609 id=e6380664-ba60-4e84-85a9-591ac73d5611 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-18609-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18609] Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18609] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18609] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18609] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18609] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:44:06 [18609] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-18609-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-18609-11) state:2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-18609-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-18609-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-18609-11-header Feb 17 18:44:06 [18621] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=18621 id=be6b31ec-2bd4-48b1-aa08-5d1e6f916030 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-18621-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18621] Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18621] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18621] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18621] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9060 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9060, Current: 9061, Stored: 9061 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9060 Feb 17 18:44:06 [18621] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9060 for localhost Feb 17 18:44:06 [18621] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9060) Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 239: var_log_free=9060 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/239) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-18621-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-18621-11) state:2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-18621-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-18621-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-18621-11-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df10 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.631 2 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.632 (null) Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=632 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9060 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/239, version=0.155.632) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 239 for var_log_free=9060 passed Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df10 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:44:06 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.632 to 0.155.631 Feb 17 18:44:06 [18633] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=18633 id=bc29dc85-1e9f-40df-a368-97b46a3ed2a7 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-18633-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18633] Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18633] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18633] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [18633] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:44:06 [18633] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:44:06 [18633] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-18633-11) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-18633-11) state:2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-18633-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-18633-11-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-18633-11-header Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18535 - exited with rc=0 Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18535:stderr [ -- empty -- ] Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18535:stdout [ arch: x86_64 ] Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18535:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18535:stdout [ free_swap: 3050 ] Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18535:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18535:stdout [ cpu_speed: 6999.99 ] Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18535:stdout [ cpu_cores: 2 ] Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18535:stdout [ cpu_load: 0.57, ] Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18535:stdout [ ram_total: 3050 ] Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18535:stdout [ ram_free: 600 ] Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18535:stdout [ root_free: 11692 ] Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18535:stdout [ root_free: 11692 ] Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18535:stdout [ var_log_free: 9060 ] Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18535:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:44:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:18535 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/939, version=0.155.632) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.632 2 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.633 (null) Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=633 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.29 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/941, version=0.155.633) Feb 17 18:44:06 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.633 to 0.155.632 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/241) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/241, version=0.155.633) Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 241 for cpu_load=0.57, passed Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.633 2 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.634 (null) Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=634 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9060 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/943, version=0.155.634) Feb 17 18:44:06 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.634 to 0.155.633 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/243) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/243, version=0.155.634) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:06 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 243 for var_log_free=9060 passed Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-mysql-wss: 2016/02/17_18:44:07 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:07 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:07 INFO: MySQL is not running Feb 17 18:44:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-metadata-agent_monitor_60000 Feb 17 18:44:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:18687 - exited with rc=0 Feb 17 18:44:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:18687:stderr [ -- empty -- ] Feb 17 18:44:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:18687:stdout [ -- empty -- ] Feb 17 18:44:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-metadata-agent action:monitor call_id:131 pid:18687 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:44:08 INFO: ntp daemon running Feb 17 18:44:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:18712 - exited with rc=0 Feb 17 18:44:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:18712:stderr [ -- empty -- ] Feb 17 18:44:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:18712:stdout [ -- empty -- ] Feb 17 18:44:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:18712 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:44:10 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(18756) = 0: Resource temporarily unavailable (11) Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:18775 - exited with rc=0 Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:18775:stderr [ -- empty -- ] Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:18775:stdout [ -- empty -- ] Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:18775 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 ocf-ns_dns: 2016/02/17_18:44:10 INFO: dnsmasq daemon running Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(18756) = 0: Resource temporarily unavailable (11) Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(18802) = 0: Resource temporarily unavailable (11) Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:18821 - exited with rc=0 Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:18821:stderr [ -- empty -- ] Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:18821:stdout [ -- empty -- ] Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:18821 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(18830) = 0: Resource temporarily unavailable (11) Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(18756) = 0: Resource temporarily unavailable (11) Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(18802) = 0: Resource temporarily unavailable (11) Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:18830 - exited with rc=0 Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:18830:stderr [ -- empty -- ] Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:18830:stdout [ -- empty -- ] Feb 17 18:44:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:18830 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:44:10 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:10 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:10 INFO: MySQL is not running Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:11 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:11 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:11 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 06e47807d42292f0f5139ae540ea7b96 for 0.155.634 (0x1733cf0 0) Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:11 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_IPaddr2: 2016/02/17_18:44:13 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.708ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.738ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.529ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.531ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:18756 - exited with rc=0 Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:18756:stderr [ -- empty -- ] Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:18756:stdout [ -- empty -- ] Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:18756 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(18802) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:44:13 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.891ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.547ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.549ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:18802 - exited with rc=0 Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:18802:stderr [ -- empty -- ] Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:18802:stdout [ -- empty -- ] Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:18802 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:18957 - exited with rc=0 Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:18957:stderr [ -- empty -- ] Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:18957:stdout [ -- empty -- ] Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:18957 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-mysql-wss: 2016/02/17_18:44:13 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:13 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:13 INFO: MySQL is not running ocf-neutron-dhcp-agent: 2016/02/17_18:44:13 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:18982 - exited with rc=0 Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:18982:stderr [ -- empty -- ] Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:18982:stdout [ -- empty -- ] Feb 17 18:44:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:18982 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:44:16 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:16 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:16 INFO: MySQL is not running Feb 17 18:44:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:44:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:44:19 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:20 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:20 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:44:21 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.592ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.616ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.530ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.538ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:19113 - exited with rc=0 Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:19113:stderr [ -- empty -- ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:19113:stdout [ -- empty -- ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:19113 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(19132) = 0: Resource temporarily unavailable (11) Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:44:21 [19246] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19246 id=750dcc1c-d63f-4272-a67b-7510877f764e Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19246-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19246] Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19246] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19246] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19246] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:44:21 [19246] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:44:21 [19246] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19246-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19246-11) state:2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19246-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19246-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19246-11-header Feb 17 18:44:21 [19251] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19251 id=710cc56e-3f72-4c1a-bd0c-7350f2248b0e Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19251-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19251] Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19251] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19251] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19251] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:44:21 [19251] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:44:21 [19251] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19251-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19251-11) state:2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19251-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19251-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19251-11-header Feb 17 18:44:21 [19267] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19267 id=91294f0c-7164-4c36-bf65-dfbc704bed4d Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19267-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19267] Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19267] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19267] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19267] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3050 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:44:21 [19267] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3050 for localhost Feb 17 18:44:21 [19267] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19267-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19267-11) state:2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19267-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19267-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19267-11-header Feb 17 18:44:21 [19275] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19275 id=7234dc94-19ab-491f-8151-c42e7a4a3377 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19275-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19275] Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19275] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19275] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19275] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:44:21 [19275] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:44:21 [19275] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19275-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19275-11) state:2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19275-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19275-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19275-11-header Feb 17 18:44:21 [19276] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19276 id=0b105cd9-74e4-4a4c-9867-df515b95272e Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19276-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19276] Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19276] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19276] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19276] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:44:21 [19276] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:44:21 [19276] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19276-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19276-11) state:2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19276-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19276-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19276-11-header Feb 17 18:44:21 [19277] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19277 id=ffef13d2-1ac4-41dc-86ab-29f59a273955 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19277-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19277] Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19277] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19277] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19277] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:44:21 [19277] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:44:21 [19277] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19277-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19277-11) state:2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19277-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19277-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19277-11-header Feb 17 18:44:21 [19282] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19282 id=bfac734a-b562-45db-b2e7-ba49f63ca773 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19282-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19282] Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19282] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19282] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19282] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.54, Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.54,, Current: 0.57,, Stored: 0.57, Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.54, Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.54,) Feb 17 18:44:21 [19282] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.54, for localhost Feb 17 18:44:21 [19282] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 245: cpu_load=0.54, Feb 17 18:44:21 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/245) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19282-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19282-11) state:2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19282-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19282-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19282-11-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349de80 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:21 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.634 2 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.635 (null) Feb 17 18:44:21 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=635 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.54, Feb 17 18:44:21 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/245, version=0.155.635) Feb 17 18:44:21 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.635 to 0.155.634 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 245 for cpu_load=0.54, passed Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349de80 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:44:21 [19287] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19287 id=f70610e1-7773-476c-ac16-d4c996ee40fc Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19287-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19287] Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19287] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19287] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19287] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:44:21 [19287] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:44:21 [19287] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19287-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19287-11) state:2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19287-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19287-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19287-11-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [19292] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19292 id=10dde26d-2681-465f-a2da-a1894d6ec5e1 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19292-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19292] Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19292] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19292] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19292] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=600 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 600, Current: 600, Stored: 600 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:21 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/945, version=0.155.635) Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:21 [19292] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=600 for localhost Feb 17 18:44:21 [19292] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19292-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19292-11) state:2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19292-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19292-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19292-11-header Feb 17 18:44:21 [19304] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19304 id=39863d31-ff60-4bc6-b0ac-482b44930294 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19304-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19304] Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19304] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19304] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19304] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:44:21 [19304] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:44:21 [19304] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19304-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19304-11) state:2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19304-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19304-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19304-11-header Feb 17 18:44:21 [19316] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19316 id=e5d54e23-7cd9-4e09-af25-8ff5c8dae8c8 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19316-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19316] Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19316] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19316] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19316] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:44:21 [19316] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:44:21 [19316] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19316-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19316-11) state:2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19316-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19316-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19316-11-header Feb 17 18:44:21 [19328] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19328 id=69a15619-f10f-45c6-903b-641e8633496e Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19328-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19328] Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19328] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19328] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19328] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9060 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9060, Current: 9060, Stored: 9060 Feb 17 18:44:21 [19328] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9060 for localhost Feb 17 18:44:21 [19328] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19328-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19328-11) state:2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19328-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19328-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19328-11-header Feb 17 18:44:21 [19340] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19340 id=f18e3b08-59ec-4455-be38-9883a3773cfe Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19340-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19340] Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19340] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19340] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [19340] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:44:21 [19340] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:44:21 [19340] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19340-11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19340-11) state:2 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19340-11-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19340-11-header Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19340-11-header Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(19132) = 0: Resource temporarily unavailable (11) Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19234 - exited with rc=0 Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19234:stderr [ -- empty -- ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19234:stdout [ arch: x86_64 ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19234:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19234:stdout [ free_swap: 3050 ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19234:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19234:stdout [ cpu_speed: 6999.99 ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19234:stdout [ cpu_cores: 2 ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19234:stdout [ cpu_load: 0.54, ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19234:stdout [ ram_total: 3050 ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19234:stdout [ ram_free: 600 ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19234:stdout [ root_free: 11692 ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19234:stdout [ root_free: 11692 ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19234:stdout [ var_log_free: 9060 ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19234:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:19234 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:44:21 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.683ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.553ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.539ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:19132 - exited with rc=0 Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:19132:stderr [ -- empty -- ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:19132:stdout [ -- empty -- ] Feb 17 18:44:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:19132 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df50 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df50 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:44:21 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.635 2 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.636 (null) Feb 17 18:44:21 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=636 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.32 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/947, version=0.155.636) Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:44:21 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.636 to 0.155.635 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:44:21 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/247) Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df50 Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:21 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:21 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/247, version=0.155.636) Feb 17 18:44:21 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 247 for cpu_load=0.54, passed Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df50 Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:21 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-mysql-wss: 2016/02/17_18:44:23 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:23 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:23 INFO: MySQL is not running Feb 17 18:44:23 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.001000 (3 ticks in 30s) Feb 17 18:44:23 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.370000 (full: 0.37 0.54 0.39 3/478 19376) Feb 17 18:44:23 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:44:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:44:24 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff Feb 17 18:44:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:19398 - exited with rc=8 Feb 17 18:44:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:19398:stderr [ -- empty -- ] Feb 17 18:44:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:19398:stdout [ vrouter ] Feb 17 18:44:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:19398:stdout [ mgmt-conntrd ] Feb 17 18:44:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:19398 exit-code:8 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:44:26 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:26 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:26 INFO: MySQL is not running Feb 17 18:44:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:44:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a6660 Feb 17 18:44:26 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:26 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: cd731fb0e2a76eeca397887da4081aa5 for 0.155.636 (0x1733cf0 0) Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a6660 Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a6660 Feb 17 18:44:26 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a6660 Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:44:28 INFO: ntp daemon running Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(19469) = 0: Resource temporarily unavailable (11) Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(19488) = 0: Resource temporarily unavailable (11) Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:19542 - exited with rc=0 Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:19542:stderr [ -- empty -- ] Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:19542:stdout [ -- empty -- ] Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:19542 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:44:28 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:44:28 INFO: net.ipv4.ip_forward = 1 Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(19469) = 0: Resource temporarily unavailable (11) Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(19488) = 0: Resource temporarily unavailable (11) Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:19553 - exited with rc=0 Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:19553:stderr [ -- empty -- ] Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:19553:stdout [ -- empty -- ] Feb 17 18:44:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:19553 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:44:29 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:29 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:29 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:44:29 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.683ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.724ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.535ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.542ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:44:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:44:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:19469 - exited with rc=0 Feb 17 18:44:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:19469:stderr [ -- empty -- ] Feb 17 18:44:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:19469:stdout [ -- empty -- ] Feb 17 18:44:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:19469 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(19488) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:44:29 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 1.041ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.531ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:44:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:19488 - exited with rc=0 Feb 17 18:44:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:19488:stderr [ -- empty -- ] Feb 17 18:44:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:19488:stdout [ -- empty -- ] Feb 17 18:44:29 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:19488 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:44:30 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:19636 - exited with rc=0 Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:19636:stderr [ -- empty -- ] Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:19636:stdout [ -- empty -- ] Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:19636 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(19663) = 0: Resource temporarily unavailable (11) Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:19669 - exited with rc=0 Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:19669:stderr [ -- empty -- ] Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:19669:stdout [ -- empty -- ] Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:19669 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_dns: 2016/02/17_18:44:30 INFO: dnsmasq daemon running Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:19663 - exited with rc=0 Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:19663:stderr [ -- empty -- ] Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:19663:stdout [ -- empty -- ] Feb 17 18:44:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:19663 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:44:32 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:32 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:32 INFO: MySQL is not running Feb 17 18:44:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:44:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:19784 - exited with rc=0 Feb 17 18:44:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:19784:stderr [ -- empty -- ] Feb 17 18:44:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:19784:stdout [ -- empty -- ] Feb 17 18:44:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:19784 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:44:34 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:44:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:19809 - exited with rc=0 Feb 17 18:44:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:19809:stderr [ -- empty -- ] Feb 17 18:44:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:19809:stdout [ -- empty -- ] Feb 17 18:44:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:19809 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:44:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:44:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:44:34 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:44:34 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:44:34 INFO: haproxy daemon running Feb 17 18:44:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(19838) = 0: Resource temporarily unavailable (11) Feb 17 18:44:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(19857) = 0: Resource temporarily unavailable (11) Feb 17 18:44:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:19876 - exited with rc=0 Feb 17 18:44:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:19876:stderr [ -- empty -- ] Feb 17 18:44:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:19876:stdout [ -- empty -- ] Feb 17 18:44:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:19876 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:44:35 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:35 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:35 INFO: MySQL is not running Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:44:36 [19960] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19960 id=c8ff95b4-bec3-4c08-af79-335ba0b3863a Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19960-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19960] Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19960] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19960] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19960] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:44:36 [19960] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:44:36 [19960] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19960-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19960-11) state:2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19960-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19960-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19960-11-header Feb 17 18:44:36 [19963] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19963 id=1fdcf91f-e781-4f89-859b-0ff4c6c1910e Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19963-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19963] Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19963] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19963] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19963] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:44:36 [19963] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:44:36 [19963] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19963-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19963-11) state:2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19963-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19963-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19963-11-header Feb 17 18:44:36 [19977] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19977 id=fbb60f4b-0e92-4b77-b337-4624786695dd Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19977-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19977] Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19977] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19977] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19977] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3050 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:44:36 [19977] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3050 for localhost Feb 17 18:44:36 [19977] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19977-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19977-11) state:2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19977-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19977-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19977-11-header Feb 17 18:44:36 [19984] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19984 id=113b8e7f-a5e7-4d59-bfe2-1eddd3f72da4 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19984-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19984] Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19984] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19984] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19984] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:44:36 [19984] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:44:36 [19984] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19984-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19984-11) state:2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19984-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19984-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19984-11-header Feb 17 18:44:36 [19985] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19985 id=d80cd0c3-72f2-4706-bee1-1b209fc0cc75 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19985-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19985] Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19985] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19985] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19985] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:44:36 [19985] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:44:36 [19985] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19985-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19985-11) state:2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19985-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19985-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19985-11-header Feb 17 18:44:36 [19986] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19986 id=2f33b58e-3ddd-4d59-8933-5256821a590d Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19986-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19986] Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19986] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19986] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19986] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:44:36 [19986] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:44:36 [19986] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19986-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19986-11) state:2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19986-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19986-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19986-11-header Feb 17 18:44:36 [19991] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19991 id=7efd2218-7f5d-40f8-9386-97dd4fa7b3f3 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19991-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19991] Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19991] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19991] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19991] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.52, Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.52,, Current: 0.54,, Stored: 0.54, Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.52, Feb 17 18:44:36 [19991] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.52, for localhost Feb 17 18:44:36 [19991] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.52,) Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 249: cpu_load=0.52, Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19991-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19991-11) state:2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19991-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19991-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19991-11-header Feb 17 18:44:36 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/249) Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e0e0 Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:36 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.636 2 Feb 17 18:44:36 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.637 (null) Feb 17 18:44:36 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=637 Feb 17 18:44:36 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.52, Feb 17 18:44:36 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/249, version=0.155.637) Feb 17 18:44:36 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.637 to 0.155.636 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:36 [19996] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 249 for cpu_load=0.52, passed Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=19996 id=e3748b4c-9d43-4cd0-921e-d3d672b8855a Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-19996-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [19996] Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19996] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19996] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [19996] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:44:36 [19996] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:44:36 [19996] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e0e0 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-19996-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-19996-11) state:2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-19996-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-19996-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-19996-11-header Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:36 [20001] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:36 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:36 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/949, version=0.155.637) Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:36 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20001 id=e16d544c-0715-4477-8cc3-617dbca3a366 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20001-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20001] Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20001] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20001] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20001] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20001] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=600 for localhost Feb 17 18:44:36 [20001] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=600 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 600, Current: 600, Stored: 600 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20001-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20001-11) state:2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20001-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20001-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20001-11-header Feb 17 18:44:36 [20013] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20013 id=cb254ac8-8296-45af-b81d-36e78e03e822 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20013-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20013] Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20013] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20013] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20013] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:44:36 [20013] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:44:36 [20013] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20013-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20013-11) state:2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20013-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20013-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20013-11-header Feb 17 18:44:36 [20025] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20025 id=b589af7f-107c-48b4-8324-3b65f37c1cd3 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20025-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20025] Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20025] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20025] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20025] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:44:36 [20025] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:44:36 [20025] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20025-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20025-11) state:2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20025-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20025-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20025-11-header Feb 17 18:44:36 [20037] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20037 id=d7a97f96-639d-4037-8405-206fe6a7533c Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20037-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20037] Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20037] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20037] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20037] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9060 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9060, Current: 9060, Stored: 9060 Feb 17 18:44:36 [20037] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9060 for localhost Feb 17 18:44:36 [20037] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20037-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20037-11) state:2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20037-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20037-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20037-11-header Feb 17 18:44:36 [20049] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20049 id=9be80e7d-5d2d-4c9e-afc4-5403509a75d8 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20049-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20049] Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20049] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20049] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [20049] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:44:36 [20049] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:44:36 [20049] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20049-11) Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20049-11) state:2 Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20049-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20049-11-header Feb 17 18:44:36 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20049-11-header Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(19838) = 0: Resource temporarily unavailable (11) Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(19857) = 0: Resource temporarily unavailable (11) Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19952 - exited with rc=0 Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19952:stderr [ -- empty -- ] Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19952:stdout [ arch: x86_64 ] Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19952:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19952:stdout [ free_swap: 3050 ] Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19952:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19952:stdout [ cpu_speed: 6999.99 ] Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19952:stdout [ cpu_cores: 2 ] Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19952:stdout [ cpu_load: 0.52, ] Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19952:stdout [ ram_total: 3050 ] Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19952:stdout [ ram_free: 600 ] Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19952:stdout [ root_free: 11692 ] Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19952:stdout [ root_free: 11692 ] Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19952:stdout [ var_log_free: 9060 ] Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:19952:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:44:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:19952 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:44:37 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.625ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.649ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.523ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.543ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:44:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:44:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:19838 - exited with rc=0 Feb 17 18:44:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:19838:stderr [ -- empty -- ] Feb 17 18:44:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:19838:stdout [ -- empty -- ] Feb 17 18:44:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:19838 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(19857) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:44:37 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.758ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:44:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:19857 - exited with rc=0 Feb 17 18:44:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:19857:stderr [ -- empty -- ] Feb 17 18:44:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:19857:stdout [ -- empty -- ] Feb 17 18:44:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:19857 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:44:38 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:38 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:38 INFO: MySQL is not running ocf-mysql-wss: 2016/02/17_18:44:41 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:41 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:41 INFO: MySQL is not running Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:41 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:41 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:41 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:41 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:41 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:41 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:41 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:41 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:41 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 41d39cdc6e236b248eb992d6f9f40443 for 0.155.637 (0x1733cf0 0) Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:41 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:41 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:41 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:41 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:41 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:41 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:41 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:41 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:44:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:44:44 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:44 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:44 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:44:45 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.700ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.723ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.539ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.543ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:44:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:44:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:20199 - exited with rc=0 Feb 17 18:44:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:20199:stderr [ -- empty -- ] Feb 17 18:44:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:20199:stdout [ -- empty -- ] Feb 17 18:44:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:20199 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(20218) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:44:45 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.768ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.544ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.538ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:44:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:20218 - exited with rc=0 Feb 17 18:44:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:20218:stderr [ -- empty -- ] Feb 17 18:44:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:20218:stdout [ -- empty -- ] Feb 17 18:44:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:20218 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:44:47 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:47 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:47 INFO: MySQL is not running Feb 17 18:44:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:44:48 INFO: ntp daemon running Feb 17 18:44:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:20387 - exited with rc=0 Feb 17 18:44:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:20387:stderr [ -- empty -- ] Feb 17 18:44:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:20387:stdout [ -- empty -- ] Feb 17 18:44:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:20387 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:44:50 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:50 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:50 INFO: MySQL is not running Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 ocf-heat-engine: 2016/02/17_18:44:50 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:20448 - exited with rc=0 Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:20448:stderr [ -- empty -- ] Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:20448:stdout [ -- empty -- ] Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:20448 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(20469) = 0: Resource temporarily unavailable (11) Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(20469) = 0: Resource temporarily unavailable (11) Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:20494 - exited with rc=0 Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:20494:stderr [ -- empty -- ] Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:20494:stdout [ -- empty -- ] Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:20494 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(20495) = 0: Resource temporarily unavailable (11) Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 ocf-ns_dns: 2016/02/17_18:44:50 INFO: dnsmasq daemon running Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(20469) = 0: Resource temporarily unavailable (11) Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(20495) = 0: Resource temporarily unavailable (11) Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:20532 - exited with rc=0 Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:20532:stderr [ -- empty -- ] Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:20532:stdout [ -- empty -- ] Feb 17 18:44:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:20532 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:44:51 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(20469) = 0: Resource temporarily unavailable (11) Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(20495) = 0: Resource temporarily unavailable (11) Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:20581 - exited with rc=8 Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:20581:stderr [ -- empty -- ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:20581:stdout [ vrouter ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:20581:stdout [ mgmt-conntrd ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:20581 exit-code:8 exec-time:0ms queue-time:0ms Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:44:51 [20613] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20613 id=809e2c8d-a046-46b5-b7b8-5079c18748cc Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20613-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20613] Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20613] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20613] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20613] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:44:51 [20613] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:44:51 [20613] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20613-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20613-11) state:2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20613-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20613-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20613-11-header Feb 17 18:44:51 [20616] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20616 id=55773eec-b9e4-42f5-872c-8a734a871708 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20616-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20616] Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20616] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20616] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20616] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:44:51 [20616] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:44:51 [20616] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20616-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20616-11) state:2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20616-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20616-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20616-11-header Feb 17 18:44:51 [20630] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20630 id=010f08d4-be86-41e9-9943-b52c5d8a668b Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20630-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20630] Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20630] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20630] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20630] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3050 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:44:51 [20630] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3050 for localhost Feb 17 18:44:51 [20630] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20630-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20630-11) state:2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20630-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20630-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20630-11-header Feb 17 18:44:51 [20637] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20637 id=d2252e5f-6c64-4cec-90dc-6831ec2f9aad Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20637-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20637] Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20637] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20637] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20637] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:44:51 [20637] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:44:51 [20637] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20637-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20637-11) state:2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20637-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20637-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20637-11-header Feb 17 18:44:51 [20638] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20638 id=273b13ac-37b0-4f6c-ac4a-fac35ee4ecfa Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20638-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20638] Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20638] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20638] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20638] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:44:51 [20638] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:44:51 [20638] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20638-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20638-11) state:2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20638-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20638-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20638-11-header Feb 17 18:44:51 [20639] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20639 id=be14dd70-c2bf-4300-b11c-013eb88f0d57 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20639-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20639] Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20639] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20639] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20639] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:44:51 [20639] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:44:51 [20639] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20639-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20639-11) state:2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20639-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20639-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20639-11-header Feb 17 18:44:51 [20644] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20644 id=86776480-f68f-47e0-8244-be363c4ad4b7 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20644-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20644] Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20644] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20644] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20644] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.49, Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.49,, Current: 0.52,, Stored: 0.52, Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.49, Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.49,) Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:44:51 [20644] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.49, for localhost Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/251) Feb 17 18:44:51 [20644] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 251: cpu_load=0.49, Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df70 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.637 2 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.638 (null) Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=638 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.49, Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/251, version=0.155.638) Feb 17 18:44:51 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.638 to 0.155.637 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 251 for cpu_load=0.49, passed Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20644-11) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20644-11) state:2 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20644-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20644-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20644-11-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df70 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/951, version=0.155.638) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:51 [20649] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20649 id=b2aa5f24-b953-4f93-8468-cc4cdbe17f3e Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20649-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20649] Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20649] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20649] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20649] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:44:51 [20649] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:44:51 [20649] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20649-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20649-11) state:2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20649-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20649-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20649-11-header Feb 17 18:44:51 [20654] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20654 id=8dc9bf4b-b5f1-42c0-90f9-f2c982920722 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20654-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20654] Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20654] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20654] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20654] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=600 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 600, Current: 600, Stored: 600 Feb 17 18:44:51 [20654] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=600 for localhost Feb 17 18:44:51 [20654] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20654-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20654-11) state:2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20654-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20654-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20654-11-header Feb 17 18:44:51 [20666] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20666 id=4e5aebb4-474c-4d9c-a6cc-724a3019a288 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20666-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20666] Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20666] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20666] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20666] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:44:51 [20666] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:44:51 [20666] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20666-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20666-11) state:2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20666-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20666-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20666-11-header Feb 17 18:44:51 [20678] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20678 id=5a9b64fe-d83a-4001-95bf-ec7dabba9e69 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20678-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20678] Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20678] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20678] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20678] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:44:51 [20678] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:44:51 [20678] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20678-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20678-11) state:2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20678-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20678-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20678-11-header Feb 17 18:44:51 [20690] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20690 id=d8d0485e-d6d7-4e5a-9553-774d167e8c65 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20690-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20690] Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20690] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20690] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20690] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9059 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9059, Current: 9060, Stored: 9060 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9059 Feb 17 18:44:51 [20690] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9059 for localhost Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9059) Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:44:51 [20690] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/253) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 253: var_log_free=9059 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d940 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.638 2 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.639 (null) Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=639 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9059 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/253, version=0.155.639) Feb 17 18:44:51 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.639 to 0.155.638 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 253 for var_log_free=9059 passed Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20690-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20690-11) state:2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20690-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20690-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20690-11-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:51 [20702] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d940 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/953, version=0.155.639) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=20702 id=73d79057-6c26-4e7f-ae4f-faa44c177184 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-20702-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [20702] Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20702] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20702] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [20702] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:44:51 [20702] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:44:51 [20702] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-20702-11) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-20702-11) state:2 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-20702-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-20702-11-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-20702-11-header Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(20469) = 0: Resource temporarily unavailable (11) Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(20495) = 0: Resource temporarily unavailable (11) Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:20605 - exited with rc=0 Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:20605:stderr [ -- empty -- ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:20605:stdout [ arch: x86_64 ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:20605:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:20605:stdout [ free_swap: 3050 ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:20605:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:20605:stdout [ cpu_speed: 6999.99 ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:20605:stdout [ cpu_cores: 2 ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:20605:stdout [ cpu_load: 0.49, ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:20605:stdout [ ram_total: 3050 ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:20605:stdout [ ram_free: 600 ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:20605:stdout [ root_free: 11692 ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:20605:stdout [ root_free: 11692 ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:20605:stdout [ var_log_free: 9059 ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:20605:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:44:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:20605 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/255) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.639 2 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.640 (null) Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=640 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.33 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/955, version=0.155.640) Feb 17 18:44:51 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.640 to 0.155.639 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/255, version=0.155.640) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 255 for cpu_load=0.49, passed Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349db20 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.640 2 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.641 (null) Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=641 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9059 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/957, version=0.155.641) Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:44:51 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.641 to 0.155.640 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/257) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349db20 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:51 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/257, version=0.155.641) Feb 17 18:44:51 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 257 for var_log_free=9059 passed Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:51 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-mysql-wss: 2016/02/17_18:44:53 ERROR: Not enough arguments [1] to ocf_log. Feb 17 18:44:53 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.001000 (3 ticks in 30s) Feb 17 18:44:53 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.230000 (full: 0.23 0.49 0.38 3/480 20725) Feb 17 18:44:53 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 ocf-mysql-wss: 2016/02/17_18:44:53 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:53 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:44:53 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.807ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.837ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.539ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.540ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:20469 - exited with rc=0 Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:20469:stderr [ -- empty -- ] Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:20469:stdout [ -- empty -- ] Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:20469 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(20495) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:44:53 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.802ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.656ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.528ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:20495 - exited with rc=0 Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:20495:stderr [ -- empty -- ] Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:20495:stdout [ -- empty -- ] Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:20495 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:20786 - exited with rc=0 Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:20786:stderr [ -- empty -- ] Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:20786:stdout [ -- empty -- ] Feb 17 18:44:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:20786 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:44:54 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:44:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:20811 - exited with rc=0 Feb 17 18:44:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:20811:stderr [ -- empty -- ] Feb 17 18:44:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:20811:stdout [ -- empty -- ] Feb 17 18:44:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:20811 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:44:56 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:56 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:56 INFO: MySQL is not running Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:56 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:56 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:56 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:56 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:56 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:56 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:56 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:56 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:44:56 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 805327d5525d4bf3a4d078daf254d099 for 0.155.641 (0x1733cf0 0) Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:56 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:56 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:56 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:44:56 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:44:56 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:56 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:56 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:44:56 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:44:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:44:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:44:58 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:44:58 INFO: net.ipv4.ip_forward = 1 Feb 17 18:44:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:44:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(20925) = 0: Resource temporarily unavailable (11) Feb 17 18:44:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:20944 - exited with rc=0 Feb 17 18:44:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:20944:stderr [ -- empty -- ] Feb 17 18:44:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:20944:stdout [ -- empty -- ] Feb 17 18:44:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:20944 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:44:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-mysql-wss: 2016/02/17_18:44:59 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:44:59 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:44:59 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:45:01 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.844ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.870ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.568ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.550ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:45:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Interrupted system call (4) Feb 17 18:45:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:20925 - exited with rc=0 Feb 17 18:45:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:20925:stderr [ -- empty -- ] Feb 17 18:45:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:20925:stdout [ -- empty -- ] Feb 17 18:45:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:20925 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(20963) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:45:01 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.820ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.540ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.529ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:45:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:45:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:20963 - exited with rc=0 Feb 17 18:45:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:20963:stderr [ -- empty -- ] Feb 17 18:45:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:20963:stdout [ -- empty -- ] Feb 17 18:45:01 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:20963 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:45:02 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:45:02 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:45:02 INFO: MySQL is not running Feb 17 18:45:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:45:04 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:45:04 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:45:04 INFO: haproxy daemon running Feb 17 18:45:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:45:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:21141 - exited with rc=0 Feb 17 18:45:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:21141:stderr [ -- empty -- ] Feb 17 18:45:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:21141:stdout [ -- empty -- ] Feb 17 18:45:04 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:21141 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:45:05 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:45:05 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:45:05 INFO: MySQL is not running Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:45:06 [21379] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=21379 id=6591d44c-4764-44d6-947a-b0a18b3fc1af Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-21379-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [21379] Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21379] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21379] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21379] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:45:06 [21379] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:45:06 [21379] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-21379-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-21379-11) state:2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-21379-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-21379-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-21379-11-header Feb 17 18:45:06 [21382] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=21382 id=1fe6b140-dd6d-4a29-b8b5-80025040d259 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-21382-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [21382] Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21382] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21382] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21382] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:45:06 [21382] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:45:06 [21382] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-21382-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-21382-11) state:2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-21382-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-21382-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-21382-11-header Feb 17 18:45:06 [21396] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=21396 id=11932422-cd71-49c5-bf8b-d23ae59c8303 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-21396-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [21396] Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21396] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21396] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21396] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21396] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3050 for localhost Feb 17 18:45:06 [21396] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3050 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-21396-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-21396-11) state:2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-21396-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-21396-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-21396-11-header Feb 17 18:45:06 [21403] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=21403 id=d70c3946-6a81-4542-bb9b-18ec27a4dea1 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-21403-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [21403] Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21403] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21403] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21403] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21403] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:45:06 [21403] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-21403-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-21403-11) state:2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-21403-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-21403-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-21403-11-header Feb 17 18:45:06 [21404] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=21404 id=171417aa-20a4-408e-a0fa-fc5e22d754a1 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-21404-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [21404] Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21404] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21404] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21404] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:45:06 [21404] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:45:06 [21404] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:06 [21405] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201c130 for uid=0 gid=0 pid=21405 id=12821ce7-d6ba-4256-b231-0d23a0681956 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-21405-12) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [21405] Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-21404-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-21404-11) state:2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-21404-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-21404-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-21404-11-header Feb 17 18:45:06 [21405] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21405] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21405] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:45:06 [21405] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:45:06 [21405] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-21405-12) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-21405-12) state:2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-21405-12-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-21405-12-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-21405-12-header Feb 17 18:45:06 [21410] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=21410 id=d7c79fce-23ae-4df6-9aef-428fe190b7b1 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-21410-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [21410] Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21410] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21410] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21410] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.47, Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.47,, Current: 0.49,, Stored: 0.49, Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.47, Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.47,) Feb 17 18:45:06 [21410] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.47, for localhost Feb 17 18:45:06 [21410] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/259) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 259: cpu_load=0.47, Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.641 2 Feb 17 18:45:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.642 (null) Feb 17 18:45:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=642 Feb 17 18:45:06 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.47, Feb 17 18:45:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/259, version=0.155.642) Feb 17 18:45:06 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.642 to 0.155.641 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 259 for cpu_load=0.47, passed Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-21410-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-21410-11) state:2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-21410-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-21410-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-21410-11-header Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df60 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:06 [21416] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:06 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:06 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/959, version=0.155.642) Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df60 Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:06 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=21416 id=c83ec1b2-fe77-43e8-b35c-dac6654b9bd5 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-21416-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [21416] Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21416] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21416] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21416] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:45:06 [21416] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:45:06 [21416] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-21416-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-21416-11) state:2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-21416-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-21416-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-21416-11-header Feb 17 18:45:06 [21423] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=21423 id=8c396d3a-6aa0-4f60-8c63-7a7eef9d5b52 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-21423-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [21423] Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21423] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21423] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21423] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=600 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 600, Current: 600, Stored: 600 Feb 17 18:45:06 [21423] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=600 for localhost Feb 17 18:45:06 [21423] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-21423-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-21423-11) state:2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-21423-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-21423-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-21423-11-header Feb 17 18:45:06 [21445] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=21445 id=5875acc4-6a29-4cff-ab5f-7831bcb626b1 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-21445-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [21445] Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21445] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21445] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21445] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:45:06 [21445] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:45:06 [21445] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-21445-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-21445-11) state:2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-21445-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-21445-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-21445-11-header Feb 17 18:45:06 [21457] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=21457 id=86f94d18-63c7-402c-887b-10a8c5bdcfd3 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-21457-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [21457] Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21457] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21457] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21457] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:45:06 [21457] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:45:06 [21457] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-21457-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-21457-11) state:2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-21457-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-21457-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-21457-11-header Feb 17 18:45:06 [21472] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=21472 id=8f336b3a-aa88-4648-a483-f367df555732 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-21472-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [21472] Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21472] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21472] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21472] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9059 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9059, Current: 9059, Stored: 9059 Feb 17 18:45:06 [21472] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9059 for localhost Feb 17 18:45:06 [21472] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-21472-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-21472-11) state:2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-21472-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-21472-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-21472-11-header Feb 17 18:45:06 [21487] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=21487 id=178a4885-6681-45d2-ac74-f4b161d40418 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-21487-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [21487] Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21487] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21487] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21487] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:06 [21487] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:45:06 [21487] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(2911) = 0: Resource temporarily unavailable (11) Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21350) = 0: Resource temporarily unavailable (11) Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:21371 - exited with rc=0 Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:21371:stderr [ -- empty -- ] Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:21371:stdout [ arch: x86_64 ] Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:21371:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:21371:stdout [ free_swap: 3050 ] Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:21371:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:21371:stdout [ cpu_speed: 6999.99 ] Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:21371:stdout [ cpu_cores: 2 ] Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:21371:stdout [ cpu_load: 0.47, ] Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:21371:stdout [ ram_total: 3050 ] Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:21371:stdout [ ram_free: 600 ] Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:21371:stdout [ root_free: 11692 ] Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:21371:stdout [ root_free: 11692 ] Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:21371:stdout [ var_log_free: 9059 ] Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:21371:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:21371 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21415) = 0: Resource temporarily unavailable (11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-21487-11) Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-21487-11) state:2 Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-21487-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-21487-11-header Feb 17 18:45:06 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-21487-11-header Feb 17 18:45:07 [6668] node-3.test.domain.local pacemaker_remoted: warning: child_timeout_callback: p_mysql_start_0 process (PID 2911) timed out Feb 17 18:45:07 [6668] node-3.test.domain.local pacemaker_remoted: warning: operation_finished: p_mysql_start_0:2911 - timed out after 300000ms Feb 17 18:45:07 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:2911:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:45:07 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:2911:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:45:07 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:2911:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:45:07 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:2911:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:45:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_start_0:2911:stdout [ -- empty -- ] Feb 17 18:45:07 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_mysql action:start call_id:132 pid:2911 exit-code:1 exec-time:300003ms queue-time:0ms Feb 17 18:45:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21350) = 0: Resource temporarily unavailable (11) Feb 17 18:45:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21415) = 0: Resource temporarily unavailable (11) Feb 17 18:45:07 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_mysql after start op Timed Out (interval=0) Feb 17 18:45:07 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed mysql-wss_meta-data_0 process 21629 exited with rc=0 Feb 17 18:45:07 [6707] node-3.test.domain.local crmd: error: process_lrm_event: Operation p_mysql_start_0: Timed Out (node=node-3.test.domain.local, call=132, timeout=300000ms) Feb 17 18:45:07 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: node-3.test.domain.local-p_mysql_start_0:132 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n ] Feb 17 18:45:07 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_mysql' with start op Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/61) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.642 2 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.643 (null) Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=643 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_mysql']/lrm_rsc_op[@id='p_mysql_last_0']: @operation_key=p_mysql_start_0, @operation=start, @transition-key=82:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=2:1;82:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=132, @rc-code=1, @op-status=2, @last-run=1455734407, @last-rc-change=1455734407, @exec-time=300003 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_mysql']/lrm_rsc_op[@id='p_mysql_last_failure_0']: @transition-key=82:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=2:1;82:89:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=132, @last-run=1455734407, @last-rc-change=1455734407 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/61, version=0.155.643) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:07 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.643 to 0.155.642 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: fail-count-p_mysql=value++ Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: value++, Current: 4, Stored: 4 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: info: attrd_local_callback: Expanded fail-count-p_mysql=value++ to 5 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of fail-count-p_mysql is 5 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-p_mysql (5) Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='fail-count-p_mysql'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[8]) Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 261: fail-count-p_mysql=5 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/261) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: last-failure-p_mysql=1455734707 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1455734707, Current: 1455734388, Stored: 1455734388 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of last-failure-p_mysql is 1455734707 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: last-failure-p_mysql (1455734707) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dbe0 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.643 2 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.644 (null) Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=644 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-fail-count-p_mysql']: @value=5 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/261, version=0.155.644) Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='last-failure-p_mysql'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[9]) Feb 17 18:45:07 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.644 to 0.155.643 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dbe0 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 263: last-failure-p_mysql=1455734707 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 261 for fail-count-p_mysql=5 passed Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/263) Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: fail-count-p_mysql=value++ Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: value++, Current: 5, Stored: 5 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: info: attrd_local_callback: Expanded fail-count-p_mysql=value++ to 6 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of fail-count-p_mysql is 6 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-p_mysql (6) Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='fail-count-p_mysql'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[8]) Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 265: fail-count-p_mysql=6 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/265) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df40 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.644 2 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.645 (null) Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=645 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-last-failure-p_mysql']: @value=1455734707 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/263, version=0.155.645) Feb 17 18:45:07 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.645 to 0.155.644 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df40 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 263 for last-failure-p_mysql=1455734707 passed Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b10 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.645 2 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.646 (null) Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=646 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-fail-count-p_mysql']: @value=6 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/265, version=0.155.646) Feb 17 18:45:07 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.646 to 0.155.645 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b10 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: last-failure-p_mysql=1455734707 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1455734707, Current: 1455734707, Stored: 1455734707 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 265 for fail-count-p_mysql=6 passed Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: fail-count-p_mysql=value++ Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: value++, Current: 6, Stored: 6 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: info: attrd_local_callback: Expanded fail-count-p_mysql=value++ to 7 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of fail-count-p_mysql is 7 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-p_mysql (7) Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='fail-count-p_mysql'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[8]) Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 267: fail-count-p_mysql=7 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/267) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df40 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.646 2 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.647 (null) Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=647 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-fail-count-p_mysql']: @value=7 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/267, version=0.155.647) Feb 17 18:45:07 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.647 to 0.155.646 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df40 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: last-failure-p_mysql=1455734707 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1455734707, Current: 1455734707, Stored: 1455734707 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 267 for fail-count-p_mysql=7 passed Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: fail-count-p_mysql=value++ Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: value++, Current: 7, Stored: 7 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: info: attrd_local_callback: Expanded fail-count-p_mysql=value++ to 8 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of fail-count-p_mysql is 8 Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-p_mysql (8) Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:07 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='fail-count-p_mysql'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[8]) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:07 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/269) Feb 17 18:45:07 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:07 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 269: fail-count-p_mysql=8 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df70 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.647 2 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.648 (null) Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=648 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-fail-count-p_mysql']: @value=8 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/269, version=0.155.648) Feb 17 18:45:08 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.648 to 0.155.647 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df70 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 269 for fail-count-p_mysql=8 passed Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/961, version=0.155.648) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/963, version=0.155.648) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/965, version=0.155.648) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/967, version=0.155.648) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-metadata-agent_monitor_60000 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/969, version=0.155.648) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: last-failure-p_mysql=1455734707 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1455734707, Current: 1455734707, Stored: 1455734707 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21350) = 0: Resource temporarily unavailable (11) Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21415) = 0: Resource temporarily unavailable (11) Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:21636 - exited with rc=0 Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:21636:stderr [ -- empty -- ] Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:21636:stdout [ -- empty -- ] Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-metadata-agent action:monitor call_id:131 pid:21636 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=103:93:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_rabbitmq-server_monitor_103000 Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=138, reply=1, notify=0, exit=4202464 Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_rabbitmq-server action:monitor call_id:138 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=102:93:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_rabbitmq-server_monitor_30000 Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=139, reply=1, notify=0, exit=4202464 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=184:93:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=ping_vip__public_monitor_20000 Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=140, reply=1, notify=0, exit=4202464 Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:ping_vip__public action:monitor call_id:140 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_mysql_stop_0 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=19:93:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_mysql_stop_0 Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=141, reply=1, notify=0, exit=4202464 Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_mysql action:stop call_id:141 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-21691-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [21691] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761349cf10, cpd=0x7f76134a4bf4 Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f761349cf10 Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-6268-21691-26-header Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-6268-21691-26-header Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-6268-21691-26-header Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-21691-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-21691-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-21691-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-21691-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-21691-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-21691-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [21691] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 ocf-mysql-wss: 2016/02/17_18:45:08 ERROR: Not enough arguments [1] to ocf_log. Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-21691-26-header Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-21691-26-header Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-21691-26-header Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:45:08 [21691] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-21691-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-21691-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-21691-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-21691-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-21691-26-header ocf-mysql-wss: 2016/02/17_18:45:08 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:45:08 INFO: MySQL is not running ocf-mysql-wss: 2016/02/17_18:45:08 DEBUG: Delete lock file: /var/lock/subsys/mysqld ocf-mysql-wss: 2016/02/17_18:45:08 DEBUG: Delete sock file: /var/run/mysqld/mysqld.sock ocf-mysql-wss: 2016/02/17_18:45:08 DEBUG: Delete pid file: /var/run/resource-agents/mysql-wss/mysql-wss.pid with content 6956 Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21350) = 0: Resource temporarily unavailable (11) Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21415) = 0: Resource temporarily unavailable (11) Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21661) = 0: Resource temporarily unavailable (11) Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21666) = 0: Resource temporarily unavailable (11) Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_stop_0:21681 - exited with rc=0 Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_stop_0:21681:stderr [ -- empty -- ] Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_stop_0:21681:stdout [ -- empty -- ] Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_mysql action:stop call_id:141 pid:21681 exit-code:0 exec-time:71ms queue-time:1ms Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_mysql after stop op complete (interval=0) Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_mysql_stop_0: ok (node=node-3.test.domain.local, call=141, rc=0, cib-update=62, confirmed=true) Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_mysql' with stop op Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/62) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.648 2 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.649 (null) Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=649 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_mysql']/lrm_rsc_op[@id='p_mysql_last_0']: @operation_key=p_mysql_stop_0, @operation=stop, @transition-key=19:93:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;19:93:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=141, @rc-code=0, @op-status=0, @last-run=1455734708, @last-rc-change=1455734708, @exec-time=71, @queue-time=1 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/62, version=0.155.649) Feb 17 18:45:08 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.649 to 0.155.648 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_mysql_start_0 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:45:08 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=89:93:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_mysql_start_0 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=142, reply=1, notify=0, exit=4202464 Feb 17 18:45:08 [6668] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_mysql action:start call_id:142 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dff0 Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dff0 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.649 2 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.650 (null) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=650 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-gtid']: @value=71dae69f-d579-11e5-a980-12eca8fb60f4:21373 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/971, version=0.155.650) Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:08 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.650 to 0.155.649 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[7]) Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/271) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/271, version=0.155.650) Feb 17 18:45:08 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 271 for gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:-1 passed Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header lrmd: 2016/02/17_18:45:08 DEBUG: p_rabbitmq-server: monitor: action start. lrmd: 2016/02/17_18:45:08 INFO: p_rabbitmq-server: get_monitor(): CHECK LEVEL IS: 30 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.650 2 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.651 (null) Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=651 Feb 17 18:45:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_mysql']: word_size:132096 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit-master'] does not exist Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: info: main: Read rabbit-master= Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-22052-14-header Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-22052-14-header Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-22052-14-header Feb 17 18:45:09 [22052] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-22052-14) Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-22052-14) state:2 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-22052-14-header Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-22052-14-header Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-22052-14-header lrmd: 2016/02/17_18:45:09 INFO: p_rabbitmq-server: get_monitor(): master attribute is 1 Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.651 2 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.652 (null) Feb 17 18:45:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=652 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='vip__public']/lrm_rsc_op[@id='vip__public_last_0']: @operation_key=vip__public_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=46:93:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;46:93:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=287, @last-run=1455734708, @last-rc-change=1455734708, @exec Feb 17 18:45:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:09 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/385, version=0.155.652) Feb 17 18:45:09 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.652 to 0.155.651 ocf-ns_IPaddr2: 2016/02/17_18:45:09 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.719ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.745ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.548ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.541ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:45:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:21350 - exited with rc=0 Feb 17 18:45:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:21350:stderr [ -- empty -- ] Feb 17 18:45:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:21350:stdout [ -- empty -- ] Feb 17 18:45:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:21350 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21415) = 0: Resource temporarily unavailable (11) Feb 17 18:45:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21661) = 0: Resource temporarily unavailable (11) Feb 17 18:45:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21666) = 0: Resource temporarily unavailable (11) Feb 17 18:45:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21764) = 0: Resource temporarily unavailable (11) lrmd: 2016/02/17_18:45:09 INFO: p_rabbitmq-server: get_monitor(): checking if rabbit app is running lrmd: 2016/02/17_18:45:09 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running. checking if we are the part of healthy cluster Feb 17 18:45:09 [22191] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:45:09 [22191] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:45:09 [22191] node-3.test.domain.local crm_node: debug: main: Attempting to process -p command for cluster type: corosync Feb 17 18:45:09 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=22191 id=d09c18f5-70b0-4c0c-ba9d-38c3b619e702 Feb 17 18:45:09 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-22191-12) Feb 17 18:45:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [22191] Feb 17 18:45:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:09 [22191] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:09 [22191] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:09 [22191] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:09 [22191] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-22191-12) Feb 17 18:45:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-22191-12) state:2 Feb 17 18:45:09 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-22191-12-header Feb 17 18:45:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-22191-12-header Feb 17 18:45:09 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-22191-12-header lrmd: 2016/02/17_18:45:09 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running. looking for master on node-2.test.domain.local Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=22198 id=3adc2d32-bf46-40ac-9f18-b858fac29a31 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-22198-14) Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [22198] Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-2.test.domain.local to 2 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='2']//transient_attributes//nvpair[@name='rabbit-master'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[25]) Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: info: main: Read rabbit-master=true Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-22198-14-header Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-22198-14) Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-22198-14-header Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-22198-14) state:2 Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-22198-14-header Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-22198-14-header Feb 17 18:45:09 [22198] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-22198-14-header Feb 17 18:45:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-22198-14-header lrmd: 2016/02/17_18:45:09 INFO: p_rabbitmq-server: get_monitor(): fetched master attribute for node-2.test.domain.local. attr value is 0 lrmd: 2016/02/17_18:45:09 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running. master is node-2.test.domain.local ocf-ns_IPaddr2: 2016/02/17_18:45:09 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.732ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.525ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.528ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:45:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:21415 - exited with rc=0 Feb 17 18:45:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:21415:stderr [ -- empty -- ] Feb 17 18:45:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:21415:stdout [ -- empty -- ] Feb 17 18:45:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:21415 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21661) = 0: Resource temporarily unavailable (11) Feb 17 18:45:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21666) = 0: Resource temporarily unavailable (11) Feb 17 18:45:09 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21764) = 0: Resource temporarily unavailable (11) lrmd: 2016/02/17_18:45:10 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running and is member of healthy cluster lrmd: 2016/02/17_18:45:10 INFO: p_rabbitmq-server: get_monitor(): preparing to update master score for node Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=22329 id=6118f3fe-9084-451e-aafb-2aac46ebd2a0 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-22329-14) Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [22329] Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit-start-time'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[24]) Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: info: main: Read rabbit-start-time=1455734428 Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-22329-14) Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-22329-14) state:2 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-22329-14-header Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-22329-14-header Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-22329-14-header Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-22329-14-header Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-22329-14-header Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-22329-14-header Feb 17 18:45:10 [22329] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:10 [22337] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:45:10 [22337] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:45:10 [22337] node-3.test.domain.local crm_node: debug: main: Attempting to process -p command for cluster type: corosync Feb 17 18:45:10 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=22337 id=fae626cc-209a-46bd-8114-a42c658582ff Feb 17 18:45:10 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-22337-12) Feb 17 18:45:10 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [22337] Feb 17 18:45:10 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [22337] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [22337] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [22337] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [22337] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:10 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-22337-12) Feb 17 18:45:10 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-22337-12) state:2 Feb 17 18:45:10 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:10 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-22337-12-header Feb 17 18:45:10 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-22337-12-header Feb 17 18:45:10 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-22337-12-header Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=22341 id=76bba1db-a1ab-44e4-96ef-18e89832d65b Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-22341-14) Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [22341] Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-2.test.domain.local to 2 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='2']//transient_attributes//nvpair[@name='rabbit-start-time'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[26]) Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: info: main: Read rabbit-start-time=1455734419 Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-22341-14) Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-22341-14) state:2 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-22341-14-header Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-22341-14-header Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-22341-14-header Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-22341-14-header Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-22341-14-header Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-22341-14-header Feb 17 18:45:10 [22341] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 lrmd: 2016/02/17_18:45:10 INFO: p_rabbitmq-server: get_monitor(): comparing our uptime (282) with node-2.test.domain.local (291) lrmd: 2016/02/17_18:45:10 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl list_channels 2>&1 > /dev/null Feb 17 18:45:10 [22417] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=22417 id=2a2a5b09-0c72-4a57-b980-b4ed86df2d63 Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-22417-11) Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [22417] Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [22417] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [22417] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [22417] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: pingd=1000 Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1000, Current: 1000, Stored: 1000 Feb 17 18:45:10 [22417] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: pingd=1000 for localhost Feb 17 18:45:10 [22417] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-22417-11) Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-22417-11) state:2 Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-22417-11-header Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-22417-11-header Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-22417-11-header Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21661) = 0: Resource temporarily unavailable (11) Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:21666 - exited with rc=0 Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:21666:stderr [ -- empty -- ] Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:21666:stdout [ -- empty -- ] Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:ping_vip__public action:monitor call_id:140 pid:21666 exit-code:0 exec-time:2033ms queue-time:0ms Feb 17 18:45:10 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource ping_vip__public after monitor op complete (interval=20000) Feb 17 18:45:10 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation ping_vip__public_monitor_20000: ok (node=node-3.test.domain.local, call=140, rc=0, cib-update=63, confirmed=false) Feb 17 18:45:10 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'ping_vip__public' with monitor op Feb 17 18:45:10 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/63) Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21764) = 0: Resource temporarily unavailable (11) Feb 17 18:45:10 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:10 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:10 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:10 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:10 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.652 2 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.653 (null) Feb 17 18:45:10 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=653 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='ping_vip__public']: &1 > /dev/null Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=22479 id=a4960cdb-09af-46fc-9911-114df2e4b683 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-22479-14) Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [22479] Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=22479 id=84630919-edd9-49dd-b07c-82fbcea09b3b Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-22479-11) Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [22479] Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_list_channels_timeouts=0 Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: 0, Stored: 0 Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_list_channels_timeouts=0 for node-3.test.domain.local Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: info: main: Update rabbit_list_channels_timeouts=0 sent via attrd Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-22479-14-header Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-22479-14-header Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-22479-14-header Feb 17 18:45:10 [22479] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-22479-11) Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-22479-11) state:2 Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-22479-11-header Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-22479-11-header Feb 17 18:45:10 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-22479-11-header Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-22479-14) Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-22479-14) state:2 Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-22479-14-header Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-22479-14-header Feb 17 18:45:10 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-22479-14-header lrmd: 2016/02/17_18:45:10 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl -q eval 'rabbit_alarm:get_alarms().' Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21661) = 0: Resource temporarily unavailable (11) Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21764) = 0: Resource temporarily unavailable (11) Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:22487 - exited with rc=0 Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:22487:stderr [ -- empty -- ] Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:22487:stdout [ -- empty -- ] Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:22487 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 ocf-ns_dns: 2016/02/17_18:45:10 INFO: dnsmasq daemon running Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21661) = 0: Resource temporarily unavailable (11) Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21764) = 0: Resource temporarily unavailable (11) Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:22573 - exited with rc=0 Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:22573:stderr [ -- empty -- ] Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:22573:stdout [ -- empty -- ] Feb 17 18:45:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:22573 exit-code:0 exec-time:0ms queue-time:0ms lrmd: 2016/02/17_18:45:11 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl -q eval 'rabbit_alarm:get_alarms().' Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=22628 id=4ac8d150-f8da-42e7-8b42-0ee5f650ff77 Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-22628-14) Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [22628] Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=22628 id=06806faa-2f6b-45f1-b647-bdef8fd4f5da Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-22628-11) Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [22628] Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_get_alarms_timeouts=0 Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: 0, Stored: 0 Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_get_alarms_timeouts=0 for node-3.test.domain.local Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: info: main: Update rabbit_get_alarms_timeouts=0 sent via attrd Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-22628-14-header Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-22628-14-header Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-22628-14-header Feb 17 18:45:11 [22628] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-22628-11) Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-22628-11) state:2 Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-22628-11-header Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-22628-11-header Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-22628-11-header Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-22628-14) Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-22628-14) state:2 Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-22628-14-header Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-22628-14-header Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-22628-14-header lrmd: 2016/02/17_18:45:11 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl -q list_queues memory messages consumer_utilisation lrmd: 2016/02/17_18:45:11 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl -q list_queues memory messages consumer_utilisation Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x182dc00 for uid=0 gid=0 pid=22761 id=50c78993-b420-410d-96b3-f93f2d688d04 Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-22761-14) Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [22761] Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=22761 id=4900000a-e672-4c74-b2a3-6013298a4ff5 Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-22761-11) Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [22761] Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_list_queues_timeouts=0 Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: 0, Stored: 0 Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_list_queues_timeouts=0 for node-3.test.domain.local Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: info: main: Update rabbit_list_queues_timeouts=0 sent via attrd Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-22761-14) Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-22761-14) state:2 Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-22761-14-header Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-22761-14-header Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-22761-14-header Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-22761-14-header Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-22761-14-header Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-22761-14-header Feb 17 18:45:11 [22761] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-22761-11) Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-22761-11) state:2 Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-22761-11-header Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-22761-11-header Feb 17 18:45:11 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-22761-11-header lrmd: 2016/02/17_18:45:11 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl -q status Feb 17 18:45:11 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:11 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:11 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:11 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:11 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:11 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.653 2 Feb 17 18:45:11 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.654 (null) Feb 17 18:45:11 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=654 Feb 17 18:45:11 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_rabbitmq-server']: word_size:263168 Feb 17 18:45:12 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:12 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:12 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:12 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:12 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:12 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:12 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:12 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:12 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:12 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:12 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:12 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:12 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.654 2 Feb 17 18:45:12 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.655 (null) Feb 17 18:45:12 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=655 Feb 17 18:45:12 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_rabbitmq-server']: Feb 17 18:45:13 [23188] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:13 [23188] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:13 [23188] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:13 [23188] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit-master'] does not exist Feb 17 18:45:13 [23188] node-3.test.domain.local crm_attribute: info: main: Read rabbit-master= Feb 17 18:45:13 [23188] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:13 [23188] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:13 [23188] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-23188-14-header Feb 17 18:45:13 [23188] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-23188-14-header Feb 17 18:45:13 [23188] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-23188-14-header Feb 17 18:45:13 [23188] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-23188-14) Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-23188-14) state:2 Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-23188-14-header Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-23188-14-header Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-23188-14-header lrmd: 2016/02/17_18:45:13 INFO: p_rabbitmq-server: get_monitor(): master attribute is 1 ocf-mysql-wss: 2016/02/17_18:45:13 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:45:13 INFO: MySQL is not running lrmd: 2016/02/17_18:45:13 INFO: p_rabbitmq-server: get_monitor(): checking if rabbit app is running lrmd: 2016/02/17_18:45:13 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running. checking if we are the part of healthy cluster Feb 17 18:45:13 [23357] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:45:13 [23357] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:45:13 [23357] node-3.test.domain.local crm_node: debug: main: Attempting to process -p command for cluster type: corosync Feb 17 18:45:13 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=23357 id=2395b3fa-0826-4193-bbe9-d1db1c767c26 Feb 17 18:45:13 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-23357-12) Feb 17 18:45:13 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [23357] Feb 17 18:45:13 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:13 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:13 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:13 [23357] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:13 [23357] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:13 [23357] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:13 [23357] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:13 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-23357-12) Feb 17 18:45:13 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-23357-12) state:2 Feb 17 18:45:13 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:13 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-23357-12-header Feb 17 18:45:13 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-23357-12-header Feb 17 18:45:13 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-23357-12-header lrmd: 2016/02/17_18:45:13 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running. looking for master on node-2.test.domain.local Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x173ec00 for uid=0 gid=0 pid=23369 id=5dc5b131-9c39-4ab5-9b4f-6d225e67d80d Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-23369-14) Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [23369] Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-2.test.domain.local to 2 Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='2']//transient_attributes//nvpair[@name='rabbit-master'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[25]) Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: info: main: Read rabbit-master=true Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-23369-14-header Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-23369-14-header Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-23369-14) Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-23369-14-header Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-23369-14) state:2 Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:13 [23369] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-23369-14-header Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-23369-14-header Feb 17 18:45:13 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-23369-14-header lrmd: 2016/02/17_18:45:13 INFO: p_rabbitmq-server: get_monitor(): fetched master attribute for node-2.test.domain.local. attr value is 0 lrmd: 2016/02/17_18:45:13 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running. master is node-2.test.domain.local Feb 17 18:45:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:45:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21764) = 0: Resource temporarily unavailable (11) Feb 17 18:45:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(22899) = 0: Resource temporarily unavailable (11) Feb 17 18:45:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:23477 - exited with rc=0 Feb 17 18:45:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:23477:stderr [ -- empty -- ] Feb 17 18:45:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:23477:stdout [ -- empty -- ] Feb 17 18:45:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:23477 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:45:14 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:45:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21764) = 0: Resource temporarily unavailable (11) Feb 17 18:45:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(22899) = 0: Resource temporarily unavailable (11) Feb 17 18:45:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:23502 - exited with rc=0 Feb 17 18:45:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:23502:stderr [ -- empty -- ] Feb 17 18:45:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:23502:stdout [ -- empty -- ] Feb 17 18:45:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:23502 exit-code:0 exec-time:0ms queue-time:0ms lrmd: 2016/02/17_18:45:14 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running and is member of healthy cluster lrmd: 2016/02/17_18:45:14 INFO: p_rabbitmq-server: get_monitor(): preparing to update master score for node Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x173ec00 for uid=0 gid=0 pid=23547 id=432bf1bf-8197-45b3-b136-9a5d5efb7e91 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-23547-14) Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [23547] Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit-start-time'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[24]) Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: info: main: Read rabbit-start-time=1455734428 Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-23547-14) Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-23547-14) state:2 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-23547-14-header Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-23547-14-header Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-23547-14-header Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-23547-14-header Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-23547-14-header Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-23547-14-header Feb 17 18:45:14 [23547] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:14 [23555] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:45:14 [23555] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:45:14 [23555] node-3.test.domain.local crm_node: debug: main: Attempting to process -p command for cluster type: corosync Feb 17 18:45:14 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=23555 id=b83b4078-dec5-4297-a753-38c7d3dc4920 Feb 17 18:45:14 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-23555-12) Feb 17 18:45:14 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [23555] Feb 17 18:45:14 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [23555] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [23555] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [23555] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [23555] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:14 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-23555-12) Feb 17 18:45:14 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-23555-12) state:2 Feb 17 18:45:14 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:14 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-23555-12-header Feb 17 18:45:14 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-23555-12-header Feb 17 18:45:14 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-23555-12-header Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x173ec00 for uid=0 gid=0 pid=23559 id=8b2508ec-3f41-4e9b-bea1-751498ba3911 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-23559-14) Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [23559] Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-2.test.domain.local to 2 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='2']//transient_attributes//nvpair[@name='rabbit-start-time'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[26]) Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: info: main: Read rabbit-start-time=1455734419 Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-23559-14) Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-23559-14) state:2 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-23559-14-header Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-23559-14-header Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-23559-14-header Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-23559-14-header Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-23559-14-header Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-23559-14-header Feb 17 18:45:14 [23559] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 lrmd: 2016/02/17_18:45:14 INFO: p_rabbitmq-server: get_monitor(): comparing our uptime (286) with node-2.test.domain.local (295) lrmd: 2016/02/17_18:45:14 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl list_channels 2>&1 > /dev/null lrmd: 2016/02/17_18:45:14 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl list_channels 2>&1 > /dev/null Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x173ec00 for uid=0 gid=0 pid=23667 id=3351e6f2-f3d1-4618-b7e5-9a064698adf7 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-23667-14) Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [23667] Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=23667 id=3aa0c7ce-8499-4459-bba8-307b8dd0c0a6 Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-23667-11) Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [23667] Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_list_channels_timeouts=0 Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: 0, Stored: 0 Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_list_channels_timeouts=0 for node-3.test.domain.local Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: info: main: Update rabbit_list_channels_timeouts=0 sent via attrd Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-23667-14) Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-23667-14) state:2 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-23667-14-header Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-23667-14-header Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-23667-14-header Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-23667-14-header Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-23667-14-header Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-23667-14-header Feb 17 18:45:14 [23667] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-23667-11) Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-23667-11) state:2 Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-23667-11-header Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-23667-11-header Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-23667-11-header Feb 17 18:45:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 lrmd: 2016/02/17_18:45:14 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl -q eval 'rabbit_alarm:get_alarms().' lrmd: 2016/02/17_18:45:14 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl -q eval 'rabbit_alarm:get_alarms().' Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x173ec00 for uid=0 gid=0 pid=23789 id=979f3cab-8c8e-43a3-b3f4-34f8932d6060 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-23789-14) Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [23789] Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=23789 id=a17dca30-bf55-400e-b0cc-2d4f495d810a Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-23789-11) Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [23789] Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_get_alarms_timeouts=0 Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: 0, Stored: 0 Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_get_alarms_timeouts=0 for node-3.test.domain.local Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: info: main: Update rabbit_get_alarms_timeouts=0 sent via attrd Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-23789-14-header Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-23789-14-header Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-23789-14-header Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-23789-14) Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-23789-14) state:2 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:14 [23789] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-23789-14-header Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-23789-14-header Feb 17 18:45:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-23789-14-header Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-23789-11) Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-23789-11) state:2 Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-23789-11-header Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-23789-11-header Feb 17 18:45:14 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-23789-11-header lrmd: 2016/02/17_18:45:14 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl -q list_queues memory messages consumer_utilisation Feb 17 18:45:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 lrmd: 2016/02/17_18:45:15 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl -q list_queues memory messages consumer_utilisation Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x173ec00 for uid=0 gid=0 pid=23943 id=c53df96f-5f5c-4ace-8fed-9ecb45d73322 Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-23943-14) Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [23943] Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:15 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=23943 id=bcdfe0d7-e7de-4bff-bf74-22a4f3d8544b Feb 17 18:45:15 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-23943-11) Feb 17 18:45:15 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [23943] Feb 17 18:45:15 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:15 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:15 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:15 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_list_queues_timeouts=0 Feb 17 18:45:15 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: 0, Stored: 0 Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_list_queues_timeouts=0 for node-3.test.domain.local Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: info: main: Update rabbit_list_queues_timeouts=0 sent via attrd Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-23943-14-header Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-23943-14-header Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-23943-14-header Feb 17 18:45:15 [23943] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:15 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-23943-11) Feb 17 18:45:15 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-23943-11) state:2 Feb 17 18:45:15 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:15 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-23943-11-header Feb 17 18:45:15 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-23943-11-header Feb 17 18:45:15 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-23943-11-header Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-23943-14) Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-23943-14) state:2 Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-23943-14-header Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-23943-14-header Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-23943-14-header lrmd: 2016/02/17_18:45:15 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl -q status lrmd: 2016/02/17_18:45:15 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl -q status lrmd: 2016/02/17_18:45:15 INFO: p_rabbitmq-server: get_monitor(): RabbitMQ is running 201 queues consuming 7.03096m of 6081m total, with 0 queued messages, average consumer utilization 1 lrmd: 2016/02/17_18:45:15 INFO: p_rabbitmq-server: get_monitor(): RabbitMQ status: [{pid,12415}, {running_applications, [{rabbitmq_management,"RabbitMQ Management Console","3.5.6"}, {rabbitmq_management_agent,"RabbitMQ Management Agent","3.5.6"}, {rabbit,"RabbitMQ","3.5.6"}, {os_mon,"CPO CXC 138 46","2.4"}, {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.5.6"}, {webmachine,"webmachine","1.10.3-rmq3.5.6-gite9359c7"}, {mochiweb,"MochiMedia Web Server","2.7.0-rmq3.5.6-git680dba8"}, {amqp_client,"RabbitMQ AMQP Client","3.5.6"}, {mnesia,"MNESIA CXC 138 12","4.13.1"}, {xmerl,"XML parser","1.3.8"}, {inets,"INETS CXC 138 49","6.0.1"}, {sasl,"SASL CXC 138 11","2.6"}, {stdlib,"ERTS CXC 138 10","2.6"}, {kernel,"ERTS CXC 138 10","4.1"}]}, {os,{unix,linux}}, {erlang_version, "Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:2:2] [async-threads:4] [kernel-poll:true]\n"}, {memory, [{total,115797664}, {connection_readers,659328}, {connection_writers,109224}, {connection_channels,1136040}, {connection_other,851432}, {queue_procs,3810848}, {queue_slave_procs,2989560}, {plugins,430144}, {other_proc,14118696}, {mnesia,912032}, {mgmt_db,11528}, {msg_index,141056}, {other_ets,1368952}, {binary,54578704}, {code,20003841}, {atom,711569}, {other_system,13964710}]}, {alarms,[]}, {listeners,[{clustering,41055,"::"},{amqp,5673,"192.168.0.4"}]}, {vm_memory_high_watermark,0.4}, {vm_memory_limit,1262442905}, {disk_free_limit,5000000}, {disk_free,12258967552}, {file_descriptors, [{total_limit,102300}, {total_used,46}, {sockets_limit,92068}, {sockets_used,44}]}, {processes,[{limit,1048576},{used,1209}]}, {run_queue,0}, {uptime,626}] lrmd: 2016/02/17_18:45:15 INFO: p_rabbitmq-server: get_monitor(): get_monitor function ready to return 0 lrmd: 2016/02/17_18:45:15 DEBUG: p_rabbitmq-server: monitor: role: lrmd: 2016/02/17_18:45:15 DEBUG: p_rabbitmq-server: monitor: result: 0 lrmd: 2016/02/17_18:45:15 DEBUG: p_rabbitmq-server: monitor: action end. Feb 17 18:45:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21764) = 0: Resource temporarily unavailable (11) Feb 17 18:45:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_monitor_30000:22899 - exited with rc=0 Feb 17 18:45:15 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:22899:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:45:15 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:22899:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:45:15 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:22899:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:45:15 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:22899:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:45:15 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:22899:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:45:15 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:22899:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:45:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_monitor_30000:22899:stdout [ -- empty -- ] Feb 17 18:45:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_rabbitmq-server action:monitor call_id:139 pid:22899 exit-code:0 exec-time:3702ms queue-time:4046ms Feb 17 18:45:15 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_rabbitmq-server after monitor op complete (interval=30000) Feb 17 18:45:15 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_rabbitmq-server_monitor_30000: ok (node=node-3.test.domain.local, call=139, rc=0, cib-update=65, confirmed=false) Feb 17 18:45:15 [6707] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_rabbitmq-server_monitor_30000:139 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n ] Feb 17 18:45:15 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_rabbitmq-server' with monitor op Feb 17 18:45:15 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/65) Feb 17 18:45:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(23674) = 0: Resource temporarily unavailable (11) Feb 17 18:45:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(23866) = 0: Resource temporarily unavailable (11) Feb 17 18:45:15 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:15 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:15 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:15 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:15 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:15 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:15 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:15 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:15 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.655 2 Feb 17 18:45:15 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.656 (null) Feb 17 18:45:15 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=656 Feb 17 18:45:15 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_rabbitmq-server']: Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=24157 id=1b954680-4fc7-4e76-98c7-8618cce311d2 Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-24157-11) Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [24157] Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: gtid=00000000-0000-0000-0000-000000000000:-1 Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 00000000-0000-0000-0000-000000000000:-1, Current: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1, Stored: 71dae69f-d579-11e5-a980-12eca8fb60f4:-1 Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of gtid is 00000000-0000-0000-0000-000000000000:-1 Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: gtid (00000000-0000-0000-0000-000000000000:-1) Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[7]) Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 273: gtid=00000000-0000-0000-0000-000000000000:-1 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/273) Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: gtid=00000000-0000-0000-0000-000000000000:-1 for node-3.test.domain.local Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: info: main: Update gtid=00000000-0000-0000-0000-000000000000:-1 sent via attrd Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-24157-14) Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-24157-14) state:2 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-24157-14-header Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-24157-14-header Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-24157-14-header Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-24157-14-header Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-24157-14-header Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-24157-14-header Feb 17 18:45:18 [24157] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-24157-11) Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-24157-11) state:2 Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-24157-11-header Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-24157-11-header Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-24157-11-header Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:18 [24159] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:45:18 [24159] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:45:18 [24159] node-3.test.domain.local crm_node: debug: main: Attempting to process -q command for cluster type: corosync Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e0a0 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.656 2 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.657 (null) Feb 17 18:45:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=657 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-gtid']: @value=00000000-0000-0000-0000-000000000000:-1 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/273, version=0.155.657) Feb 17 18:45:18 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.657 to 0.155.656 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-24159-28) Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [24159] Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7f76134a26a0 Feb 17 18:45:18 [24159] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [24159] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:18 [24159] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7f76134a26a0 Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:18 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 273 for gtid=00000000-0000-0000-0000-000000000000:-1 passed Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e0a0 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7f76134a26a0 Feb 17 18:45:18 [24159] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:18 [24159] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-request-6268-24159-28-header Feb 17 18:45:18 [24159] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-response-6268-24159-28-header Feb 17 18:45:18 [24159] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-event-6268-24159-28-header Feb 17 18:45:18 [24159] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-24159-28) Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-24159-28) state:2 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7f76134a26a0 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-6268-24159-28-header Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-6268-24159-28-header Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-6268-24159-28-header Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:18 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/973, version=0.155.657) Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:18 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:18 [24168] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:45:18 [24168] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:45:18 [24168] node-3.test.domain.local crm_node: debug: main: Attempting to process -l command for cluster type: corosync Feb 17 18:45:18 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=24168 id=28211159-4fd8-457f-80e0-85bb247381fb Feb 17 18:45:18 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-24168-12) Feb 17 18:45:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [24168] Feb 17 18:45:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:18 [24168] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:18 [24168] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:18 [24168] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:18 [24168] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-24168-12) Feb 17 18:45:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-24168-12) state:2 Feb 17 18:45:18 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-24168-12-header Feb 17 18:45:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-24168-12-header Feb 17 18:45:18 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-24168-12-header Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x173ec00 for uid=0 gid=0 pid=24172 id=0aed3de0-9812-4d5d-8b6c-b3045e24c364 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-24172-14) Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [24172] Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: unpack_config: Stop all active resources: false Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: unpack_config: Default stickiness: 0 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: determine_op_status: p_mysql_monitor_60000 on node-2.test.domain.local returned 'unknown error' (1) instead of the expected value: 'ok' (0) Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:1 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:1 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:1 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op start for p_mysql:1 on node-3.test.domain.local: unknown error (1) Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:1 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:1 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-3.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-3.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-3.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-3.test.domain.local to ping_vip__public:1 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-3.test.domain.local to p_ntp:1 Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-24172-14) Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-24172-14) state:2 Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-24172-14-header Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-24172-14-header Feb 17 18:45:18 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-24172-14-header Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-24172-14-header Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-24172-14-header Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-24172-14-header Feb 17 18:45:18 [24172] node-3.test.domain.local crm_resource: info: crm_xml_cleanup: Cleaning up memory from libxml2 ocf-mysql-wss: 2016/02/17_18:45:18 INFO: Starting MySQL Feb 17 18:45:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:45:18 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff Feb 17 18:45:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21764) = 0: Resource temporarily unavailable (11) Feb 17 18:45:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:24223 - exited with rc=8 Feb 17 18:45:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:24223:stderr [ -- empty -- ] Feb 17 18:45:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:24223:stdout [ vrouter ] Feb 17 18:45:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:24223:stdout [ mgmt-conntrd ] Feb 17 18:45:18 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:24223 exit-code:8 exec-time:0ms queue-time:0ms Feb 17 18:45:21 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:45:21 [25027] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=25027 id=63a97c92-2cbe-41a3-a1cf-619697b4a11e Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-25027-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [25027] Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25027] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25027] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25027] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:45:22 [25027] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:45:22 [25027] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25027-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25027-11) state:2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25027-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25027-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25027-11-header Feb 17 18:45:22 [25037] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=25037 id=14b25846-ff4b-4012-8011-41d2dd6d1965 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-25037-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [25037] Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25037] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25037] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25037] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:45:22 [25037] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:45:22 [25037] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25037-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25037-11) state:2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25037-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25037-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25037-11-header Feb 17 18:45:22 [25055] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=25055 id=d99b3a6f-5c99-42ee-8c4a-7c891eb15168 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-25055-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [25055] Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25055] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25055] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25055] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3000 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3000, Current: 3050, Stored: 3050 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of free_swap is 3000 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: free_swap (3000) Feb 17 18:45:22 [25055] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3000 for localhost Feb 17 18:45:22 [25055] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='free_swap'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[13]) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 275: free_swap=3000 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/275) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25055-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25055-11) state:2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25055-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25055-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25055-11-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df60 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.657 2 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.658 (null) Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=658 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-free_swap']: @value=3000 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/275, version=0.155.658) Feb 17 18:45:22 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.658 to 0.155.657 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 275 for free_swap=3000 passed Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df60 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [25063] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/975, version=0.155.658) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=25063 id=4a6dfcb9-6f54-44cf-95ac-3ea353fc428c Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-25063-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [25063] Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25063] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25063] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25063] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:45:22 [25063] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:45:22 [25063] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25063-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25063-11) state:2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25063-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25063-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25063-11-header Feb 17 18:45:22 [25064] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=25064 id=68af4087-94b5-4151-8f0d-c6131b20a7c1 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-25064-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [25064] Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25064] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25064] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25064] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:45:22 [25064] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:45:22 [25064] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25064-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25064-11) state:2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25064-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25064-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25064-11-header Feb 17 18:45:22 [25065] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=25065 id=de7e29f8-3608-4509-b0f9-6e6f04bcfb04 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-25065-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [25065] Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25065] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25065] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25065] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:45:22 [25065] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:45:22 [25065] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25065-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25065-11) state:2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25065-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25065-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25065-11-header Feb 17 18:45:22 [25070] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=25070 id=7ab3014e-65fe-4e91-965a-1b7b082fcc12 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-25070-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [25070] Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25070] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25070] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25070] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.54, Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.54,, Current: 0.47,, Stored: 0.47, Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.54, Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.54,) Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:22 [25070] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.54, for localhost Feb 17 18:45:22 [25070] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 277: cpu_load=0.54, Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/277) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25070-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25070-11) state:2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25070-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25070-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25070-11-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [25075] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349da00 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.658 2 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.659 (null) Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=659 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.54, Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/277, version=0.155.659) Feb 17 18:45:22 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.659 to 0.155.658 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 277 for cpu_load=0.54, passed Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=25075 id=6a83c9ea-178f-45fc-9ed2-72f29b5f39ef Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-25075-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [25075] Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349da00 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [25075] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25075] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25075] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e0f0 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/977, version=0.155.659) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e0f0 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.659 2 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.660 (null) Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=660 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.35 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/979, version=0.155.660) Feb 17 18:45:22 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.660 to 0.155.659 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/279) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:45:22 [25075] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:45:22 [25075] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25075-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25075-11) state:2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25075-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25075-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25075-11-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [25080] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/279, version=0.155.660) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=25080 id=30f5b8ec-5b0f-4588-9941-77922d6caf9e Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-25080-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [25080] Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 279 for cpu_load=0.54, passed Feb 17 18:45:22 [25080] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25080] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25080] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=600 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 600, Current: 600, Stored: 600 Feb 17 18:45:22 [25080] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=600 for localhost Feb 17 18:45:22 [25080] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25080-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25080-11) state:2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25080-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25080-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25080-11-header Feb 17 18:45:22 [25092] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=25092 id=82f8b450-4842-481f-866b-a5d2ab11001b Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-25092-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [25092] Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25092] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25092] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25092] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:45:22 [25092] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:45:22 [25092] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25092-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25092-11) state:2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25092-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25092-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25092-11-header Feb 17 18:45:22 [25104] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=25104 id=e4f4aaf1-38b2-4220-adc3-f8528ebdec63 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-25104-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [25104] Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25104] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25104] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25104] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11692 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11692, Current: 11692, Stored: 11692 Feb 17 18:45:22 [25104] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11692 for localhost Feb 17 18:45:22 [25104] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25104-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25104-11) state:2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25104-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25104-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25104-11-header Feb 17 18:45:22 [25116] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=25116 id=4c59ed1e-45b2-471e-a521-12e41fa3066f Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-25116-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [25116] Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25116] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25116] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25116] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9058 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9058, Current: 9059, Stored: 9059 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9058 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9058) Feb 17 18:45:22 [25116] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9058 for localhost Feb 17 18:45:22 [25116] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 281: var_log_free=9058 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d8f0 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.660 2 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.661 (null) Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=661 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9058 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/981, version=0.155.661) Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/281) Feb 17 18:45:22 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.661 to 0.155.660 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d8f0 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:22 [25128] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a6760 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a6760 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.661 2 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.662 (null) Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=662 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9058 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/281, version=0.155.662) Feb 17 18:45:22 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.662 to 0.155.661 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 283: var_log_free=9058 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 281 for var_log_free=9058 passed Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/283) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d7c0 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201c130 for uid=0 gid=0 pid=25128 id=f3f10c1d-c201-402e-a909-5391ad20a60f Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-25128-12) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [25128] Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25116-11) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25116-11) state:2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25116-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25116-11-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25116-11-header Feb 17 18:45:22 [25128] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25128] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [25128] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:45:22 [25128] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:45:22 [25128] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25128-12) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25128-12) state:2 Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25128-12-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25128-12-header Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25128-12-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21764) = 0: Resource temporarily unavailable (11) Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25019 - exited with rc=0 Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25019:stderr [ -- empty -- ] Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25019:stdout [ arch: x86_64 ] Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25019:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25019:stdout [ free_swap: 3000 ] Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25019:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25019:stdout [ cpu_speed: 6999.99 ] Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25019:stdout [ cpu_cores: 2 ] Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25019:stdout [ cpu_load: 0.54, ] Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25019:stdout [ ram_total: 3050 ] Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25019:stdout [ ram_free: 600 ] Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25019:stdout [ root_free: 11692 ] Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25019:stdout [ root_free: 11692 ] Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25019:stdout [ var_log_free: 9058 ] Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25019:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:25019 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d7c0 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/283, version=0.155.662) Feb 17 18:45:22 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 283 for var_log_free=9058 passed Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:22 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:22 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:22 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/983, version=0.155.662) Feb 17 18:45:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:45:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:45:23 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.004667 (14 ticks in 30s) Feb 17 18:45:23 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 0.570000 (full: 0.57 0.54 0.40 1/492 25400) Feb 17 18:45:23 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 ocf-mysql-wss: 2016/02/17_18:45:23 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:45:23 INFO: MySQL is not running ocf-ns_IPaddr2: 2016/02/17_18:45:25 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.752ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.772ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.546ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.557ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:45:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21764) = 0: Interrupted system call (4) Feb 17 18:45:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:25227 - exited with rc=0 Feb 17 18:45:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:25227:stderr [ -- empty -- ] Feb 17 18:45:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:25227:stdout [ -- empty -- ] Feb 17 18:45:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:25227 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:25 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(25378) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:45:26 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 1.626ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.526ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.545ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(21764) = 0: Resource temporarily unavailable (11) Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:25378 - exited with rc=0 Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:25378:stderr [ -- empty -- ] Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:25378:stdout [ -- empty -- ] Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:25378 exit-code:0 exec-time:0ms queue-time:0ms ocf-mysql-wss: 2016/02/17_18:45:26 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:45:26 SST IS IN PROGRESS: ocf-mysql-wss: 2016/02/17_18:45:26 INFO: MySQL started ocf-mysql-wss: 2016/02/17_18:45:26 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:45:26 SST IS IN PROGRESS: ocf-mysql-wss: 2016/02/17_18:45:26 ERROR: GTID have wrong format: : ocf-mysql-wss: 2016/02/17_18:45:26 INFO: Wrong GTID: :, Removing Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x173ec00 for uid=0 gid=0 pid=25547 id=96ae3816-ff6f-4bb6-989f-39126fbec204 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-25547-14) Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [25547] Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=25547 id=c54c1b20-4427-4574-a4c9-81023f32dfbd Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-25547-11) Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [25547] Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: gtid= Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: (null), Current: 00000000-0000-0000-0000-000000000000:-1, Stored: 00000000-0000-0000-0000-000000000000:-1 Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: gtid () Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[7]) Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: gtid=(null) for node-3.test.domain.local Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: info: main: Update gtid= sent via attrd Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-25547-14-header Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-25547-14-header Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-25547-14-header Feb 17 18:45:26 [25547] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent delete 285: node=3, attr=gtid, id=, set=(null), section=status Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_delete operation for section status to master (origin=local/attrd/285) Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-25547-14) Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-25547-14) state:2 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-25547-14-header Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-25547-14-header Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-25547-14-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e050 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e050 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_start_0:21764 - exited with rc=0 Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:21764:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:21764:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:21764:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:21764:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:21764:stderr [ Warning: Using a password on the command line interface can be insecure. ] Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:21764:stderr [ ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111) ] Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:21764:stderr [ Warning: Using a password on the command line interface can be insecure. ] Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:21764:stderr [ ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111) ] Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_start_0:21764:stdout [ -- empty -- ] Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_mysql action:start call_id:142 pid:21764 exit-code:0 exec-time:18256ms queue-time:0ms Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_mysql after start op complete (interval=0) Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.662 2 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.663 (null) Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: -- /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-gtid'] Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=663 Feb 17 18:45:26 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.663 to 0.155.662 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_delete operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/285, version=0.155.663) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/985, version=0.155.663) Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] does not exist Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for gtid=(null) passed Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 285 for gtid=(null) passed Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25547-11) Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25547-11) state:2 Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25547-11-header Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25547-11-header Feb 17 18:45:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25547-11-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: info: services_os_action_execute: Managed mysql-wss_meta-data_0 process 25548 exited with rc=0 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/66) Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_mysql_start_0: ok (node=node-3.test.domain.local, call=142, rc=0, cib-update=66, confirmed=true) Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_mysql_start_0:142 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\nWarning: Using a password on the command line interface can be insecure.\nERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)\nWa Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_mysql' with start op Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.663 2 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.664 (null) Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=664 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_mysql']/lrm_rsc_op[@id='p_mysql_last_0']: @operation_key=p_mysql_start_0, @operation=start, @transition-key=89:93:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;89:93:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=142, @exec-time=18256, @queue-time=0 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/66, version=0.155.664) Feb 17 18:45:26 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.664 to 0.155.663 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=95:94:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_mysql_monitor_60000 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 33b25540-7772-40d4-b8de-ed3f21e9e5ee: rc=143, reply=1, notify=0, exit=4202464 Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_mysql action:monitor call_id:143 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header ocf-mysql-wss: 2016/02/17_18:45:26 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:45:26 SST IS IN PROGRESS: Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_60000:25556 - exited with rc=0 Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_60000:25556:stderr [ -- empty -- ] Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_60000:25556:stdout [ -- empty -- ] Feb 17 18:45:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_mysql action:monitor call_id:143 pid:25556 exit-code:0 exec-time:27ms queue-time:0ms Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_mysql after monitor op complete (interval=60000) Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_mysql_monitor_60000: ok (node=node-3.test.domain.local, call=143, rc=0, cib-update=67, confirmed=false) Feb 17 18:45:26 [6707] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_mysql' with monitor op Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/67) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.664 2 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.665 (null) Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=665 Feb 17 18:45:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_mysql']: word_size:263168 Feb 17 18:45:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:29 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:29 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:29 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:29 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:29 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:29 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:29 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:29 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:29 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:29 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:29 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:29 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.665 2 Feb 17 18:45:29 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.666 (null) Feb 17 18:45:29 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=666 Feb 17 18:45:29 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='vip__public']: word_size:33792 Feb 17 18:45:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:32 [25829] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:32 [25829] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:32 [25829] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:32 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: pingd=1000 Feb 17 18:45:32 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1000, Current: 1000, Stored: 1000 Feb 17 18:45:32 [25829] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: pingd=1000 for localhost Feb 17 18:45:32 [25829] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:32 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25829-11) Feb 17 18:45:32 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25829-11) state:2 Feb 17 18:45:32 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25829-11-header Feb 17 18:45:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25829-11-header Feb 17 18:45:32 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25829-11-header Feb 17 18:45:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:25680 - exited with rc=0 Feb 17 18:45:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:25680:stderr [ -- empty -- ] Feb 17 18:45:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:25680:stdout [ -- empty -- ] Feb 17 18:45:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:ping_vip__public action:monitor call_id:140 pid:25680 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(25718) = 0: Resource temporarily unavailable (11) Feb 17 18:45:32 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(25784) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:45:33 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.751ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.783ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.543ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.536ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:45:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:25718 - exited with rc=0 Feb 17 18:45:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:25718:stderr [ -- empty -- ] Feb 17 18:45:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:25718:stdout [ -- empty -- ] Feb 17 18:45:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:25718 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(25784) = 0: Resource temporarily unavailable (11) Feb 17 18:45:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:45:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(25784) = 0: Resource temporarily unavailable (11) Feb 17 18:45:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:25861 - exited with rc=0 Feb 17 18:45:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:25861:stderr [ -- empty -- ] Feb 17 18:45:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:25861:stdout [ -- empty -- ] Feb 17 18:45:33 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:25861 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:45:34 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:45:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(25784) = 0: Resource temporarily unavailable (11) Feb 17 18:45:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:25887 - exited with rc=0 Feb 17 18:45:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:25887:stderr [ -- empty -- ] Feb 17 18:45:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:25887:stdout [ -- empty -- ] Feb 17 18:45:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:25887 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:45:34 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.849ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.537ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.533ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:45:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:25784 - exited with rc=0 Feb 17 18:45:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:25784:stderr [ -- empty -- ] Feb 17 18:45:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:25784:stdout [ -- empty -- ] Feb 17 18:45:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:25784 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:34 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:34 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:34 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: d930007bc0ada069766ce08fa6ef48e2 for 0.155.666 (0x1733cf0 0) Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:34 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:34 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:34 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:34 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:45:34 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:45:34 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:45:35 INFO: haproxy daemon running Feb 17 18:45:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:25931 - exited with rc=0 Feb 17 18:45:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:25931:stderr [ -- empty -- ] Feb 17 18:45:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:25931:stdout [ -- empty -- ] Feb 17 18:45:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:25931 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:37 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:37 [25997] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.666 2 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.667 (null) Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=667 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-free_swap']: @value=2700 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:37 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.667 to 0.155.666 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/987, version=0.155.667) Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='free_swap'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[12]) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/288) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=25997 id=437c3a98-8347-411e-972f-21173a027397 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-25997-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [25997] Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [25997] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [25997] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [25997] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:45:37 [25997] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:37 [25997] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/288, version=0.155.667) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 288 for free_swap=3000 passed Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-25997-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-25997-11) state:2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-25997-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-25997-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-25997-11-header Feb 17 18:45:37 [26002] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=26002 id=4acecd4d-d682-45df-8759-c0cdb433396b Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-26002-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [26002] Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26002] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26002] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26002] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:45:37 [26002] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:45:37 [26002] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-26002-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-26002-11) state:2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-26002-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-26002-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-26002-11-header Feb 17 18:45:37 [26016] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=26016 id=110fb407-5495-4c13-962c-cf2897aba119 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-26016-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [26016] Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26016] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26016] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26016] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3000 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3000, Current: 3000, Stored: 3000 Feb 17 18:45:37 [26016] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3000 for localhost Feb 17 18:45:37 [26016] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-26016-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-26016-11) state:2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-26016-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-26016-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-26016-11-header Feb 17 18:45:37 [26023] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=26023 id=923fb1d0-5858-457c-aeab-92c6ad714f5a Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-26023-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [26023] Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26023] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26023] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26023] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.667 2 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.668 (null) Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=668 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.38 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/989, version=0.155.668) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.668 to 0.155.667 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349ddb0 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:37 [26023] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:37 [26023] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-26023-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-26023-11) state:2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-26023-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-26023-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-26023-11-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/290) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349ddb0 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/290, version=0.155.668) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 290 for cpu_load=0.54, passed Feb 17 18:45:37 [26025] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=26025 id=15070461-9101-4a5b-bb27-919ab403ec28 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-26025-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [26025] Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26025] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26025] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26025] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:45:37 [26025] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:45:37 [26025] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-26025-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-26025-11) state:2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-26025-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-26025-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-26025-11-header Feb 17 18:45:37 [26026] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=26026 id=9b5831e8-3439-4bdb-b82f-82462af81fd6 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-26026-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [26026] Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26026] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26026] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26026] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:45:37 [26026] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:45:37 [26026] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-26026-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-26026-11) state:2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-26026-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-26026-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-26026-11-header Feb 17 18:45:37 [26033] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=26033 id=8a6e524b-2861-459a-baa7-e1c6f5ba3cf7 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-26033-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [26033] Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26033] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26033] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26033] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.63, Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.63,, Current: 0.54,, Stored: 0.54, Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.63, Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.63,) Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:45:37 [26033] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.63, for localhost Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 292: cpu_load=0.63, Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/292) Feb 17 18:45:37 [26033] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-26033-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-26033-11) state:2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-26033-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-26033-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-26033-11-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [26038] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d970 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.668 2 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.669 (null) Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=669 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.63, Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/292, version=0.155.669) Feb 17 18:45:37 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.669 to 0.155.668 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 292 for cpu_load=0.63, passed Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=26038 id=74bdd18d-99bf-4e09-b998-50f0106a15ba Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-26038-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [26038] Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:45:37 [26038] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26038] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26038] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:45:37 [26038] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:45:37 [26038] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d970 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-26038-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-26038-11) state:2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-26038-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-26038-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-26038-11-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:37 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:37 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/991, version=0.155.669) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:37 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:37 [26043] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=26043 id=3966ad7b-6937-4d95-a139-116f26a9850b Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-26043-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [26043] Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26043] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26043] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [26043] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=600 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 600, Current: 600, Stored: 600 Feb 17 18:45:37 [26043] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=600 for localhost Feb 17 18:45:37 [26043] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-26043-11) Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-26043-11) state:2 Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-26043-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-26043-11-header Feb 17 18:45:37 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-26043-11-header Feb 17 18:45:38 [26057] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=26057 id=46c4d25e-3d9b-4cb7-a2f0-fe3b28542c07 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-26057-11) Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [26057] Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [26057] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [26057] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [26057] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11691 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11691, Current: 11692, Stored: 11692 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of root_free is 11691 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: root_free (11691) Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[19]) Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 294: root_free=11691 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/294) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.669 2 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.670 (null) Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=670 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-root_free']: @value=11691 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/294, version=0.155.670) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:38 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.670 to 0.155.669 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 294 for root_free=11691 passed Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [26057] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11691 for localhost Feb 17 18:45:38 [26057] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-26057-11) Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-26057-11) state:2 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-26057-11-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-26057-11-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-26057-11-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/993, version=0.155.670) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:38 [26071] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=26071 id=9f2ce12b-6062-4352-8e81-2e906e9b6d58 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-26071-11) Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [26071] Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [26071] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [26071] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [26071] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11691 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11691, Current: 11691, Stored: 11691 Feb 17 18:45:38 [26071] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11691 for localhost Feb 17 18:45:38 [26071] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-26071-11) Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-26071-11) state:2 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-26071-11-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-26071-11-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-26071-11-header Feb 17 18:45:38 [26087] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=26087 id=e74ddfc2-750d-489d-a9c6-434bfd5808cc Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-26087-11) Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [26087] Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [26087] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [26087] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [26087] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9057 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9057, Current: 9058, Stored: 9058 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9057 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9057) Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/296) Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 296: var_log_free=9057 Feb 17 18:45:38 [26087] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9057 for localhost Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [26087] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dfb0 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.670 2 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.671 (null) Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=671 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9057 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/296, version=0.155.671) Feb 17 18:45:38 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.671 to 0.155.670 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 296 for var_log_free=9057 passed Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-26087-11) Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-26087-11) state:2 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-26087-11-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-26087-11-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-26087-11-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dfb0 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/995, version=0.155.671) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:38 [26106] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=26106 id=94928ddf-f80c-4672-abb7-32b71aa195aa Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-26106-11) Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [26106] Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [26106] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [26106] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [26106] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18774 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18774, Current: 18424, Stored: 18424 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_lib_mysql_free is 18774 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_lib_mysql_free (18774) Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_lib_mysql_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/298) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 298: var_lib_mysql_free=18774 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [26106] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18774 for localhost Feb 17 18:45:38 [26106] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-26106-11) Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-26106-11) state:2 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-26106-11-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-26106-11-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-26106-11-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25988 - exited with rc=0 Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25988:stderr [ -- empty -- ] Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25988:stdout [ arch: x86_64 ] Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25988:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25988:stdout [ free_swap: 3000 ] Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25988:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25988:stdout [ cpu_speed: 6999.99 ] Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25988:stdout [ cpu_cores: 2 ] Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25988:stdout [ cpu_load: 0.63, ] Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25988:stdout [ ram_total: 3050 ] Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25988:stdout [ ram_free: 600 ] Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25988:stdout [ root_free: 11691 ] Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25988:stdout [ root_free: 11691 ] Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25988:stdout [ var_log_free: 9057 ] Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:25988:stdout [ var_lib_mysql_free: 18774 ] Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:25988 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.671 2 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.672 (null) Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=672 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_lib_mysql_free']: @value=18774 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/298, version=0.155.672) Feb 17 18:45:38 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.672 to 0.155.671 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:38 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 298 for var_lib_mysql_free=18774 passed Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:38 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:38 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/997, version=0.155.672) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:38 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:45:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:45:42 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.698ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.729ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.543ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.536ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:45:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:26115 - exited with rc=0 Feb 17 18:45:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:26115:stderr [ -- empty -- ] Feb 17 18:45:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:26115:stdout [ -- empty -- ] Feb 17 18:45:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:26115 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(26139) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:45:42 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 2.714ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.713ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.537ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:45:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:26139 - exited with rc=0 Feb 17 18:45:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:26139:stderr [ -- empty -- ] Feb 17 18:45:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:26139:stdout [ -- empty -- ] Feb 17 18:45:42 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:26139 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:43 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:43 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:43 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d8b0 Feb 17 18:45:43 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:43 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:43 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:43 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:43 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:43 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: f5afea207edc2bace2a1b09a2eea42a4 for 0.155.672 (0x1733cf0 0) Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d8b0 Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:43 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:43 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:43 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:43 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:43 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:43 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:43 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:43 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:45:45 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff Feb 17 18:45:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_rabbitmq-server_monitor_30000 Feb 17 18:45:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:26272 - exited with rc=8 Feb 17 18:45:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:26272:stderr [ -- empty -- ] Feb 17 18:45:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:26272:stdout [ vrouter ] Feb 17 18:45:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:26272:stdout [ mgmt-conntrd ] Feb 17 18:45:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:26272 exit-code:8 exec-time:0ms queue-time:0ms Feb 17 18:45:45 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(26296) = 0: Resource temporarily unavailable (11) Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-26315-26) Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [26315] Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761349cf10, cpd=0x7f76134a6b34 Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f761349cf10 Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-6268-26315-26-header Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-6268-26315-26-header Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-6268-26315-26-header Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-26315-26) Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-26315-26) state:2 Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-26315-26-header Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-26315-26-header Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-26315-26-header Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-26315-26) Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [26315] Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-26315-26-header Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-26315-26-header Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-26315-26-header Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:45:46 [26315] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-26315-26) Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-26315-26) state:2 Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-26315-26-header Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-26315-26-header Feb 17 18:45:46 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-26315-26-header lrmd: 2016/02/17_18:45:46 DEBUG: p_rabbitmq-server: monitor: action start. Feb 17 18:45:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 lrmd: 2016/02/17_18:45:47 INFO: p_rabbitmq-server: get_monitor(): CHECK LEVEL IS: 0 Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:47 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:47 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:47 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:47 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:47 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:47 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:47 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:47 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:47 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:47 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:47 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:47 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:47 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:47 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:47 [6707] node-3.test.domain.local crmd: debug: throttle_update: Host node-2.test.domain.local supports a maximum of 4 jobs and throttle mode 0001. New job limit is 2 Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:47 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:45:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:45:48 INFO: ntp daemon running Feb 17 18:45:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(26296) = 0: Resource temporarily unavailable (11) Feb 17 18:45:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(26415) = 0: Resource temporarily unavailable (11) Feb 17 18:45:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(26518) = 0: Resource temporarily unavailable (11) Feb 17 18:45:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:26595 - exited with rc=0 Feb 17 18:45:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:26595:stderr [ -- empty -- ] Feb 17 18:45:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:26595:stdout [ -- empty -- ] Feb 17 18:45:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:26595 exit-code:0 exec-time:0ms queue-time:0ms lrmd: 2016/02/17_18:45:48 INFO: p_rabbitmq-server: get_monitor(): get_status() returns 0. lrmd: 2016/02/17_18:45:48 INFO: p_rabbitmq-server: get_monitor(): also checking if we are master. Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1439bb0 for uid=0 gid=0 pid=26733 id=2859b51f-ecfd-4d1a-9d77-d6ebdbd1a9b9 Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-26733-14) Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [26733] Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit-master'] does not exist Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: info: main: Read rabbit-master= Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-26733-14) Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-26733-14) state:2 Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-26733-14-header Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-26733-14-header Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-26733-14-header Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-26733-14-header Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-26733-14-header Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-26733-14-header Feb 17 18:45:49 [26733] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 lrmd: 2016/02/17_18:45:49 INFO: p_rabbitmq-server: get_monitor(): master attribute is 1 lrmd: 2016/02/17_18:45:49 INFO: p_rabbitmq-server: get_monitor(): checking if rabbit app is running lrmd: 2016/02/17_18:45:49 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running. checking if we are the part of healthy cluster Feb 17 18:45:49 [26847] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:45:49 [26847] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:45:49 [26847] node-3.test.domain.local crm_node: debug: main: Attempting to process -p command for cluster type: corosync Feb 17 18:45:49 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=26847 id=a9bff458-541f-4dbc-b0d5-ed3b2931c21c Feb 17 18:45:49 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-26847-12) Feb 17 18:45:49 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [26847] Feb 17 18:45:49 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:49 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:49 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:49 [26847] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:49 [26847] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:49 [26847] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:49 [26847] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:49 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-26847-12) Feb 17 18:45:49 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-26847-12) state:2 Feb 17 18:45:49 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:49 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-26847-12-header Feb 17 18:45:49 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-26847-12-header Feb 17 18:45:49 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-26847-12-header lrmd: 2016/02/17_18:45:49 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running. looking for master on node-2.test.domain.local Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1439bb0 for uid=0 gid=0 pid=26854 id=3a1d8208-ca6b-4b32-961e-a959a8e6469a Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-26854-14) Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [26854] Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-2.test.domain.local to 2 Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='2']//transient_attributes//nvpair[@name='rabbit-master'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[25]) Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: info: main: Read rabbit-master=true Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-26854-14-header Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-26854-14) Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-26854-14) state:2 Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-26854-14-header Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-26854-14-header Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-26854-14-header Feb 17 18:45:49 [26854] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-26854-14-header Feb 17 18:45:49 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-26854-14-header lrmd: 2016/02/17_18:45:49 INFO: p_rabbitmq-server: get_monitor(): fetched master attribute for node-2.test.domain.local. attr value is 0 lrmd: 2016/02/17_18:45:49 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running. master is node-2.test.domain.local ocf-ns_IPaddr2: 2016/02/17_18:45:50 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.611ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.641ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.544ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.550ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(26296) = 0: Resource temporarily unavailable (11) Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:26415 - exited with rc=0 Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:26415:stderr [ -- empty -- ] Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:26415:stdout [ -- empty -- ] Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:26415 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(26518) = 0: Resource temporarily unavailable (11) Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:45:50 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(26296) = 0: Resource temporarily unavailable (11) Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(26518) = 0: Resource temporarily unavailable (11) Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:26995 - exited with rc=0 Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:26995:stderr [ -- empty -- ] Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:26995:stdout [ -- empty -- ] Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:26995 exit-code:0 exec-time:0ms queue-time:0ms lrmd: 2016/02/17_18:45:50 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running and is member of healthy cluster lrmd: 2016/02/17_18:45:50 INFO: p_rabbitmq-server: get_monitor(): preparing to update master score for node Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1439bb0 for uid=0 gid=0 pid=27045 id=bf369d0b-3697-464e-9b53-a3342c02bc32 Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-27045-14) Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [27045] Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit-start-time'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[23]) Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: info: main: Read rabbit-start-time=1455734428 Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-27045-14) Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-27045-14) state:2 Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-27045-14-header Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-27045-14-header Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-27045-14-header Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-27045-14-header Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-27045-14-header Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-27045-14-header Feb 17 18:45:50 [27045] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:50 [27053] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:45:50 [27053] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:45:50 [27053] node-3.test.domain.local crm_node: debug: main: Attempting to process -p command for cluster type: corosync Feb 17 18:45:50 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=27053 id=b5136e51-c211-4fe0-9d47-fa754e87f83d Feb 17 18:45:50 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-27053-12) Feb 17 18:45:50 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [27053] Feb 17 18:45:50 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:50 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:50 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:50 [27053] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:50 [27053] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:50 [27053] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:50 [27053] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:50 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-27053-12) Feb 17 18:45:50 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-27053-12) state:2 Feb 17 18:45:50 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:50 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-27053-12-header Feb 17 18:45:50 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-27053-12-header Feb 17 18:45:50 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-27053-12-header Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1439bb0 for uid=0 gid=0 pid=27057 id=3c3f6118-917b-4ac5-a09d-98aa34fe7d25 Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-27057-14) Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [27057] Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-2.test.domain.local to 2 Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='2']//transient_attributes//nvpair[@name='rabbit-start-time'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[26]) Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: info: main: Read rabbit-start-time=1455734419 Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-27057-14) Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-27057-14) state:2 Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-27057-14-header Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-27057-14-header Feb 17 18:45:50 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-27057-14-header Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-27057-14-header Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-27057-14-header Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-27057-14-header Feb 17 18:45:50 [27057] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 lrmd: 2016/02/17_18:45:50 INFO: p_rabbitmq-server: get_monitor(): comparing our uptime (322) with node-2.test.domain.local (331) lrmd: 2016/02/17_18:45:50 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl list_channels 2>&1 > /dev/null Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(26296) = 0: Resource temporarily unavailable (11) Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(26518) = 0: Resource temporarily unavailable (11) Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:27078 - exited with rc=0 Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:27078:stderr [ -- empty -- ] Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:27078:stdout [ -- empty -- ] Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:27078 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 ocf-ns_IPaddr2: 2016/02/17_18:45:50 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.606ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.711ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.539ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(26296) = 0: Resource temporarily unavailable (11) Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:26518 - exited with rc=0 Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:26518:stderr [ -- empty -- ] Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:26518:stdout [ -- empty -- ] Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:26518 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(27159) = 0: Resource temporarily unavailable (11) ocf-ns_dns: 2016/02/17_18:45:50 INFO: dnsmasq daemon running Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(26296) = 0: Resource temporarily unavailable (11) Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:27159 - exited with rc=0 Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:27159:stderr [ -- empty -- ] Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:27159:stdout [ -- empty -- ] Feb 17 18:45:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:27159 exit-code:0 exec-time:0ms queue-time:0ms lrmd: 2016/02/17_18:45:51 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl list_channels 2>&1 > /dev/null Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1439bb0 for uid=0 gid=0 pid=27210 id=4bdf38f8-2c99-404b-9425-bfff015f7267 Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-27210-14) Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [27210] Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27210 id=139c4701-c545-4b54-983b-db8e0e0aff15 Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27210-11) Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27210] Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_list_channels_timeouts=0 Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: 0, Stored: 0 Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_list_channels_timeouts=0 for node-3.test.domain.local Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: info: main: Update rabbit_list_channels_timeouts=0 sent via attrd Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-27210-14) Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-27210-14) state:2 Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-27210-14-header Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-27210-14-header Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-27210-14-header Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-27210-14-header Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-27210-14-header Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-27210-14-header Feb 17 18:45:51 [27210] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27210-11) Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27210-11) state:2 Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27210-11-header Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27210-11-header Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27210-11-header lrmd: 2016/02/17_18:45:51 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl -q eval 'rabbit_alarm:get_alarms().' lrmd: 2016/02/17_18:45:51 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl -q eval 'rabbit_alarm:get_alarms().' Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1439bb0 for uid=0 gid=0 pid=27313 id=9b5847b8-6bbf-44ba-a80c-ba181e51228f Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-27313-14) Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [27313] Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27313 id=eca22e8f-a05d-4c8f-b468-5f46c42bf7b1 Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27313-11) Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27313] Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_get_alarms_timeouts=0 Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: 0, Stored: 0 Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_get_alarms_timeouts=0 for node-3.test.domain.local Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: info: main: Update rabbit_get_alarms_timeouts=0 sent via attrd Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-27313-14-header Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-27313-14) Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-27313-14-header Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-27313-14) state:2 Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-27313-14-header Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-27313-14-header Feb 17 18:45:51 [27313] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-27313-14-header Feb 17 18:45:51 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-27313-14-header Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27313-11) Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27313-11) state:2 Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27313-11-header Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27313-11-header Feb 17 18:45:51 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27313-11-header lrmd: 2016/02/17_18:45:51 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl -q list_queues memory messages consumer_utilisation Feb 17 18:45:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of ping_vip__public_monitor_20000 lrmd: 2016/02/17_18:45:52 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl -q list_queues memory messages consumer_utilisation Feb 17 18:45:52 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1439bb0 for uid=0 gid=0 pid=27465 id=6ab57b21-f9ff-427e-a66e-4bf0296c1cfd Feb 17 18:45:52 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-27465-14) Feb 17 18:45:52 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [27465] Feb 17 18:45:52 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:52 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:52 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:45:52 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:52 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27465 id=db049d2e-2295-4185-9248-c33e10e6299d Feb 17 18:45:52 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27465-11) Feb 17 18:45:52 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27465] Feb 17 18:45:52 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:52 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:52 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:52 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_list_queues_timeouts=0 Feb 17 18:45:52 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: 0, Stored: 0 Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_list_queues_timeouts=0 for node-3.test.domain.local Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: info: main: Update rabbit_list_queues_timeouts=0 sent via attrd Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:52 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-27465-14) Feb 17 18:45:52 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-27465-14) state:2 Feb 17 18:45:52 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:52 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-27465-14-header Feb 17 18:45:52 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-27465-14-header Feb 17 18:45:52 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-27465-14-header Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-27465-14-header Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-27465-14-header Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-27465-14-header Feb 17 18:45:52 [27465] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:52 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27465-11) Feb 17 18:45:52 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27465-11) state:2 Feb 17 18:45:52 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:52 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27465-11-header Feb 17 18:45:52 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27465-11-header Feb 17 18:45:52 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27465-11-header lrmd: 2016/02/17_18:45:52 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl -q status lrmd: 2016/02/17_18:45:53 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl -q status Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e000 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.672 2 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.673 (null) Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=673 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.46 Feb 17 18:45:53 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.673 to 0.155.672 lrmd: 2016/02/17_18:45:53 INFO: p_rabbitmq-server: get_monitor(): RabbitMQ is running 201 queues consuming 6.83186m of 6081m total, with 0 queued messages, average consumer utilization 1 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/999, version=0.155.673) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/300) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e000 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] lrmd: 2016/02/17_18:45:53 INFO: p_rabbitmq-server: get_monitor(): RabbitMQ status: [{pid,12415}, {running_applications, [{rabbitmq_management,"RabbitMQ Management Console","3.5.6"}, {rabbitmq_management_agent,"RabbitMQ Management Agent","3.5.6"}, {rabbit,"RabbitMQ","3.5.6"}, {os_mon,"CPO CXC 138 46","2.4"}, {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.5.6"}, {webmachine,"webmachine","1.10.3-rmq3.5.6-gite9359c7"}, {mochiweb,"MochiMedia Web Server","2.7.0-rmq3.5.6-git680dba8"}, {amqp_client,"RabbitMQ AMQP Client","3.5.6"}, {mnesia,"MNESIA CXC 138 12","4.13.1"}, {xmerl,"XML parser","1.3.8"}, {inets,"INETS CXC 138 49","6.0.1"}, {sasl,"SASL CXC 138 11","2.6"}, {stdlib,"ERTS CXC 138 10","2.6"}, {kernel,"ERTS CXC 138 10","4.1"}]}, {os,{unix,linux}}, {erlang_version, "Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:2:2] [async-threads:4] [kernel-poll:true]\n"}, {memory, [{total,115476880}, {connection_readers,620976}, {connection_writers,113904}, {connection_channels,1198376}, {connection_other,851432}, {queue_procs,3716584}, {queue_slave_procs,2899040}, {plugins,430144}, {other_proc,14001472}, {mnesia,912032}, {mgmt_db,11528}, {msg_index,141056}, {other_ets,1368872}, {binary,54524408}, {code,20003841}, {atom,711569}, {other_system,13971646}]}, {alarms,[]}, {listeners,[{clustering,41055,"::"},{amqp,5673,"192.168.0.4"}]}, {vm_memory_high_watermark,0.4}, {vm_memory_limit,1262442905}, {disk_free_limit,5000000}, {disk_free,12258893824}, {file_descriptors, [{total_limit,102300}, {total_used,46}, {sockets_limit,92068}, {sockets_used,44}]}, {processes,[{limit,1048576},{used,1209}]}, {run_queue,0}, {uptime,663}] Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 lrmd: 2016/02/17_18:45:53 INFO: p_rabbitmq-server: get_monitor(): get_monitor function ready to return 0 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/300, version=0.155.673) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 300 for cpu_load=0.63, passed Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header lrmd: 2016/02/17_18:45:53 DEBUG: p_rabbitmq-server: monitor: role: Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header lrmd: 2016/02/17_18:45:53 DEBUG: p_rabbitmq-server: monitor: result: 0 lrmd: 2016/02/17_18:45:53 DEBUG: p_rabbitmq-server: monitor: action end. Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_monitor_30000:26296 - exited with rc=0 Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:26296:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:26296:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:26296:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:26296:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:26296:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:26296:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_monitor_30000:26296:stdout [ -- empty -- ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_rabbitmq-server action:monitor call_id:139 pid:26296 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(27428) = 0: Resource temporarily unavailable (11) Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.003333 (10 ticks in 30s) Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 2.190000 (full: 2.19 0.91 0.53 2/502 27629) Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: throttle_handle_load: Noticable CPU load detected: 2.190000 Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: throttle_timer_cb: New throttle mode: 0001 (was 0000) Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: info: throttle_send_command: Updated throttle state to 0001 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a1b10 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-28) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f76134a2650 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.673 2 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.674 (null) Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=674 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9057 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1001, version=0.155.674) Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:45:53 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.674 to 0.155.673 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/302) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a1b10 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-28-header Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-28-header Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-28-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-28) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-28) state:2 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f76134a2650 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-28-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-28-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-28-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:45:53 [6707] node-3.test.domain.local crmd: debug: throttle_update: Host node-3.test.domain.local supports a maximum of 4 jobs and throttle mode 0001. New job limit is 2 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/302, version=0.155.674) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 302 for var_log_free=9057 passed Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:45:53 [27638] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27638 id=d3e0bb30-0049-43bc-8b99-2294a1d979f9 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27638-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27638] Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27638] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27638] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27638] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:45:53 [27638] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:45:53 [27638] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27638-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27638-11) state:2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27638-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27638-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27638-11-header Feb 17 18:45:53 [27641] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27641 id=1a9cff38-68d9-471e-aa91-a892b41c565d Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27641-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27641] Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27641] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27641] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27641] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27641] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:45:53 [27641] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27641-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27641-11) state:2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27641-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27641-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27641-11-header Feb 17 18:45:53 [27655] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27655 id=969fa66d-b4e2-4890-b219-c910c13f7725 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27655-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27655] Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27655] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27655] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27655] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3000 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3000, Current: 3000, Stored: 3000 Feb 17 18:45:53 [27655] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3000 for localhost Feb 17 18:45:53 [27655] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27655-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27655-11) state:2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27655-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27655-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27655-11-header Feb 17 18:45:53 [27662] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27662 id=324c0358-cac2-4ae0-98b5-046dd031a3bc Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27662-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27662] Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27662] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27662] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27662] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:45:53 [27662] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:45:53 [27662] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27662-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27662-11) state:2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27662-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27662-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27662-11-header Feb 17 18:45:53 [27663] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27663 id=7ed5805b-1948-48e4-acc8-16b2c7f458ff Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27663-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27663] Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27663] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27663] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27663] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:45:53 [27663] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:45:53 [27663] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27663-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27663-11) state:2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27663-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27663-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27663-11-header Feb 17 18:45:53 [27664] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27664 id=94485173-7093-43ad-82e4-ad69226da96d Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27664-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27664] Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27664] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27664] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27664] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:45:53 [27664] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:45:53 [27664] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27664-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27664-11) state:2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27664-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27664-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27664-11-header Feb 17 18:45:53 [27669] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27669 id=d538a693-39fe-40f9-90cb-b3b03d771c3c Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27669-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27669] Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27669] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27669] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27669] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.91, Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.91,, Current: 0.63,, Stored: 0.63, Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.91, Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.91,) Feb 17 18:45:53 [27669] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.91, for localhost Feb 17 18:45:53 [27669] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 304: cpu_load=0.91, Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/304) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27669-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27669-11) state:2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27669-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27669-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27669-11-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e000 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e000 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.674 2 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.675 (null) Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=675 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.91, Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/304, version=0.155.675) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 304 for cpu_load=0.91, passed Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:45:53 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.675 to 0.155.674 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:45:53 [27674] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27674 id=da7d602b-0ce8-44d5-b71b-de1460b3719b Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27674-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27674] Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27674] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27674] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27674] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:45:53 [27674] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:45:53 [27674] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27674-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27674-11) state:2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27674-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27674-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27674-11-header Feb 17 18:45:53 [27680] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1003, version=0.155.675) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27680 id=1b8a0c89-8605-4c8e-aed4-3f674617b68b Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27680-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27680] Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27680] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27680] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:53 [27680] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=600 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 600, Current: 600, Stored: 600 Feb 17 18:45:53 [27680] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=600 for localhost Feb 17 18:45:53 [27680] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27680-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27680-11) state:2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27680-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27680-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27680-11-header Feb 17 18:45:53 [27693] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27693 id=a8770079-24a6-4393-b4d0-ba6f364a19ff Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27693-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27693] Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27693] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27693] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27693] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11691 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11691, Current: 11691, Stored: 11691 Feb 17 18:45:53 [27693] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11691 for localhost Feb 17 18:45:53 [27693] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27693-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27693-11) state:2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27693-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27693-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27693-11-header Feb 17 18:45:53 [27705] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27705 id=fe5d8e4d-3d8e-4c4d-8f06-b6a2fbd450b3 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27705-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27705] Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27705] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27705] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27705] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11691 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11691, Current: 11691, Stored: 11691 Feb 17 18:45:53 [27705] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11691 for localhost Feb 17 18:45:53 [27705] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27705-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27705-11) state:2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27705-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27705-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27705-11-header Feb 17 18:45:53 [27717] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27717 id=ac18294e-62f3-4983-af11-bb5b3480eb5a Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27717-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27717] Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27717] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27717] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27717] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9057 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9057, Current: 9057, Stored: 9057 Feb 17 18:45:53 [27717] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9057 for localhost Feb 17 18:45:53 [27717] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27717-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27717-11) state:2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27717-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27717-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27717-11-header Feb 17 18:45:53 [27729] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27729 id=1918e800-8687-4ca3-b504-4c9c92674945 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27729-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27729] Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27729] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27729] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [27729] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18662 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18662, Current: 18774, Stored: 18774 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_lib_mysql_free is 18662 Feb 17 18:45:53 [27729] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18662 for localhost Feb 17 18:45:53 [27729] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_lib_mysql_free (18662) Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_lib_mysql_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/306) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 306: var_lib_mysql_free=18662 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27729-11) Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27729-11) state:2 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27729-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27729-11-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27729-11-header Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(27428) = 0: Resource temporarily unavailable (11) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349e000 Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:27630 - exited with rc=0 Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:27630:stderr [ -- empty -- ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:27630:stdout [ arch: x86_64 ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:27630:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:27630:stdout [ free_swap: 3000 ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:27630:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:27630:stdout [ cpu_speed: 6999.99 ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:27630:stdout [ cpu_cores: 2 ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:27630:stdout [ cpu_load: 0.91, ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:27630:stdout [ ram_total: 3050 ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:27630:stdout [ ram_free: 600 ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:27630:stdout [ root_free: 11691 ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:27630:stdout [ root_free: 11691 ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:27630:stdout [ var_log_free: 9057 ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:27630:stdout [ var_lib_mysql_free: 18662 ] Feb 17 18:45:53 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:27630 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.675 2 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.676 (null) Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=676 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_lib_mysql_free']: @value=18662 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/306, version=0.155.676) Feb 17 18:45:53 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.676 to 0.155.675 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349e000 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:45:53 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 306 for var_lib_mysql_free=18662 passed Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:53 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:53 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1005, version=0.155.676) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:53 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(27428) = 0: Resource temporarily unavailable (11) Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:27758 - exited with rc=0 Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:27758:stderr [ -- empty -- ] Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:27758:stdout [ -- empty -- ] Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:27758 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:45:54 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(27428) = 0: Resource temporarily unavailable (11) Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:27787 - exited with rc=0 Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:27787:stderr [ -- empty -- ] Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:27787:stdout [ -- empty -- ] Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:27787 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:54 [27826] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:45:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=27826 id=c451b76f-518b-480f-9b0b-0585d940fe85 Feb 17 18:45:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-27826-11) Feb 17 18:45:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [27826] Feb 17 18:45:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:54 [27826] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:54 [27826] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:54 [27826] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:45:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: pingd=1000 Feb 17 18:45:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1000, Current: 1000, Stored: 1000 Feb 17 18:45:54 [27826] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: pingd=1000 for localhost Feb 17 18:45:54 [27826] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:45:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-27826-11) Feb 17 18:45:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-27826-11) state:2 Feb 17 18:45:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:45:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-27826-11-header Feb 17 18:45:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-27826-11-header Feb 17 18:45:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-27826-11-header Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:27428 - exited with rc=0 Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:27428:stderr [ -- empty -- ] Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:27428:stdout [ -- empty -- ] Feb 17 18:45:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:ping_vip__public action:monitor call_id:140 pid:27428 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:45:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:45:58 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 1.268ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 1.291ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.547ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.543ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:45:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:27846 - exited with rc=0 Feb 17 18:45:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:27846:stderr [ -- empty -- ] Feb 17 18:45:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:27846:stdout [ -- empty -- ] Feb 17 18:45:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:27846 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:45:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(27889) = 0: Resource temporarily unavailable (11) Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:58 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:58 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:45:58 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 95f84e7f313a59348115915c010e2eab for 0.155.676 (0x1733cf0 0) Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:58 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:45:58 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:45:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:58 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:45:58 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:45:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:45:58 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:45:58 INFO: net.ipv4.ip_forward = 1 Feb 17 18:45:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(27889) = 0: Resource temporarily unavailable (11) Feb 17 18:45:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:28025 - exited with rc=0 Feb 17 18:45:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:28025:stderr [ -- empty -- ] Feb 17 18:45:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:28025:stdout [ -- empty -- ] Feb 17 18:45:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:28025 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:45:58 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.781ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.534ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:45:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:27889 - exited with rc=0 Feb 17 18:45:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:27889:stderr [ -- empty -- ] Feb 17 18:45:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:27889:stdout [ -- empty -- ] Feb 17 18:45:58 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:27889 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:46:03 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:46:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:46:05 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:46:05 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:46:05 INFO: haproxy daemon running Feb 17 18:46:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(28256) = 0: Resource temporarily unavailable (11) Feb 17 18:46:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(28275) = 0: Resource temporarily unavailable (11) Feb 17 18:46:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:28429 - exited with rc=0 Feb 17 18:46:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:28429:stderr [ -- empty -- ] Feb 17 18:46:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:28429:stdout [ -- empty -- ] Feb 17 18:46:05 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:28429 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:46:06 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.602ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.635ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.537ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.538ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:46:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:28256 - exited with rc=0 Feb 17 18:46:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:28256:stderr [ -- empty -- ] Feb 17 18:46:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:28256:stdout [ -- empty -- ] Feb 17 18:46:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:28256 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:06 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(28275) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:46:07 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.667ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.534ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:46:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:28275 - exited with rc=0 Feb 17 18:46:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:28275:stderr [ -- empty -- ] Feb 17 18:46:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:28275:stdout [ -- empty -- ] Feb 17 18:46:07 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:28275 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-metadata-agent_monitor_60000 Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:28504 - exited with rc=0 Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:28504:stderr [ -- empty -- ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_60000:28504:stdout [ -- empty -- ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-metadata-agent action:monitor call_id:131 pid:28504 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349da20 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349da20 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.676 2 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.677 (null) Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=677 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.48 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1007, version=0.155.677) Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:08 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.677 to 0.155.676 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/308) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/308, version=0.155.677) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 308 for cpu_load=0.91, passed Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d7c0 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.677 2 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.678 (null) Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=678 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-root_free']: @value=11690 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1009, version=0.155.678) Feb 17 18:46:08 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.678 to 0.155.677 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d7c0 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[19]) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/310) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/310, version=0.155.678) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 310 for root_free=11691 passed Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 ocf-ns_ntp: 2016/02/17_18:46:08 INFO: ntp daemon running Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:28548 - exited with rc=0 Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:28548:stderr [ -- empty -- ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:28548:stdout [ -- empty -- ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:28548 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(28551) = 0: Resource temporarily unavailable (11) Feb 17 18:46:08 [28567] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=28567 id=5f8156ed-9247-4524-a60e-1562025452c2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-28567-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [28567] Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28567] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28567] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28567] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:46:08 [28567] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:46:08 [28567] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-28567-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-28567-11) state:2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-28567-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-28567-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-28567-11-header Feb 17 18:46:08 [28570] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=28570 id=58cbf755-72cd-4e50-9e88-632c57499ef5 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-28570-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [28570] Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28570] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28570] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28570] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:46:08 [28570] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:46:08 [28570] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-28570-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-28570-11) state:2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-28570-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-28570-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-28570-11-header Feb 17 18:46:08 [28584] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=28584 id=58317365-d90d-4cbd-8db4-c0f4b742b2a6 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-28584-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [28584] Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28584] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28584] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28584] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=2850 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2850, Current: 3000, Stored: 3000 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of free_swap is 2850 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: free_swap (2850) Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='free_swap'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[12]) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 312: free_swap=2850 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/312) Feb 17 18:46:08 [28584] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=2850 for localhost Feb 17 18:46:08 [28584] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-28584-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-28584-11) state:2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-28584-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-28584-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-28584-11-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349ded0 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.678 2 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.679 (null) Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=679 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-free_swap']: @value=2850 Feb 17 18:46:08 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.679 to 0.155.678 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349ded0 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/312, version=0.155.679) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 312 for free_swap=2850 passed Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:08 [28591] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=28591 id=4a712a21-74aa-4ad1-b744-19b1cc567527 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-28591-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [28591] Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28591] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28591] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28591] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:46:08 [28591] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:46:08 [28591] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-28591-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-28591-11) state:2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-28591-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-28591-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-28591-11-header Feb 17 18:46:08 [28592] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=28592 id=1f69c579-3f06-49d8-ae7c-2c5dbf5a8f87 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-28592-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [28592] Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28592] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28592] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28592] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:46:08 [28592] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:46:08 [28592] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-28592-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-28592-11) state:2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-28592-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-28592-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-28592-11-header Feb 17 18:46:08 [28593] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=28593 id=451037f6-8c3c-46db-a009-e1fa8753f17a Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-28593-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [28593] Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28593] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28593] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28593] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:46:08 [28593] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:46:08 [28593] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-28593-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-28593-11) state:2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-28593-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-28593-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-28593-11-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1011, version=0.155.679) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [28598] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=28598 id=fc3d4b10-621c-4dff-b31b-0cf6e843cc4f Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-28598-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [28598] Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28598] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28598] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28598] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.87, Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.87,, Current: 0.91,, Stored: 0.91, Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.87, Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.87,) Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 314: cpu_load=0.87, Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/314) Feb 17 18:46:08 [28598] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.87, for localhost Feb 17 18:46:08 [28598] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-28598-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-28598-11) state:2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-28598-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-28598-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-28598-11-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [28603] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349de40 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=28603 id=cc1d40e4-66f8-4a3d-bd54-3a0cc5f0d745 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-28603-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [28603] Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:08 [28603] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28603] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28603] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:46:08 [28603] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:46:08 [28603] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-28603-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-28603-11) state:2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-28603-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-28603-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-28603-11-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.679 2 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.680 (null) Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=680 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.87, Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/314, version=0.155.680) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349de40 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:46:08 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.680 to 0.155.679 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 314 for cpu_load=0.87, passed Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1013, version=0.155.680) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [28608] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=28608 id=43e39794-624a-4eec-b798-3af8b83bb1c0 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-28608-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [28608] Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28608] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28608] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28608] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=600 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 600, Current: 600, Stored: 600 Feb 17 18:46:08 [28608] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=600 for localhost Feb 17 18:46:08 [28608] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-28608-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-28608-11) state:2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-28608-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-28608-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-28608-11-header Feb 17 18:46:08 [28620] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=28620 id=270b8cbf-e1e4-41fa-9ec0-73220b0c5ce5 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-28620-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [28620] Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28620] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28620] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28620] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11691 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11691, Current: 11691, Stored: 11691 Feb 17 18:46:08 [28620] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11691 for localhost Feb 17 18:46:08 [28620] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-28620-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-28620-11) state:2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-28620-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-28620-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-28620-11-header Feb 17 18:46:08 [28632] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=28632 id=3ce93a19-3483-486f-818d-cd922acdd059 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-28632-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [28632] Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28632] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28632] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28632] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11691 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11691, Current: 11691, Stored: 11691 Feb 17 18:46:08 [28632] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11691 for localhost Feb 17 18:46:08 [28632] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-28632-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-28632-11) state:2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-28632-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-28632-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-28632-11-header Feb 17 18:46:08 [28644] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=28644 id=08a45c98-5fc8-4ee6-bbfe-638a12577402 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-28644-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [28644] Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28644] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28644] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28644] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9056 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9056, Current: 9057, Stored: 9057 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9056 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9056) Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 316: var_log_free=9056 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/316) Feb 17 18:46:08 [28644] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9056 for localhost Feb 17 18:46:08 [28644] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-28644-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-28644-11) state:2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-28644-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-28644-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-28644-11-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349da20 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.680 2 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.681 (null) Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=681 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9056 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/316, version=0.155.681) Feb 17 18:46:08 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.681 to 0.155.680 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 316 for var_log_free=9056 passed Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349da20 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:08 [28656] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=28656 id=0c5cefaa-9480-437c-acb7-43848f1a782f Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-28656-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [28656] Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28656] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28656] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [28656] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18422 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18422, Current: 18662, Stored: 18662 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_lib_mysql_free is 18422 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_lib_mysql_free (18422) Feb 17 18:46:08 [28656] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18422 for localhost Feb 17 18:46:08 [28656] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_lib_mysql_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[21]) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 318: var_lib_mysql_free=18422 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-28656-11) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-28656-11) state:2 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-28656-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-28656-11-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-28656-11-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/318) Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:28551 - exited with rc=0 Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:28551:stderr [ -- empty -- ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:28551:stdout [ arch: x86_64 ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:28551:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:28551:stdout [ free_swap: 2850 ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:28551:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:28551:stdout [ cpu_speed: 6999.99 ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:28551:stdout [ cpu_cores: 2 ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:28551:stdout [ cpu_load: 0.87, ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:28551:stdout [ ram_total: 3050 ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:28551:stdout [ ram_free: 600 ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:28551:stdout [ root_free: 11691 ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:28551:stdout [ root_free: 11691 ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:28551:stdout [ var_log_free: 9056 ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:28551:stdout [ var_lib_mysql_free: 18422 ] Feb 17 18:46:08 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:28551 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df60 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df60 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.681 2 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.682 (null) Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=682 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_lib_mysql_free']: @value=18422 Feb 17 18:46:08 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.682 to 0.155.681 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/318, version=0.155.682) Feb 17 18:46:08 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 318 for var_lib_mysql_free=18422 passed Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1015, version=0.155.682) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:08 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1017, version=0.155.682) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:08 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:09 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349da60 Feb 17 18:46:09 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:09 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:09 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:46:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:46:09 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:46:09 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.682 2 Feb 17 18:46:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.683 (null) Feb 17 18:46:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=683 Feb 17 18:46:09 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-gtid']: @value=71dae69f-d579-11e5-a980-12eca8fb60f4:21507 Feb 17 18:46:09 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1019, version=0.155.683) Feb 17 18:46:09 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:09 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.683 to 0.155.682 Feb 17 18:46:09 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] does not exist Feb 17 18:46:09 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for gtid=(null) passed Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349da60 Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:46:09 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:46:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:46:10 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:46:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:28681 - exited with rc=0 Feb 17 18:46:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:28681:stderr [ -- empty -- ] Feb 17 18:46:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:28681:stdout [ -- empty -- ] Feb 17 18:46:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:28681 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:46:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:28708 - exited with rc=0 Feb 17 18:46:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:28708:stderr [ -- empty -- ] Feb 17 18:46:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:28708:stdout [ -- empty -- ] Feb 17 18:46:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:28708 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:10 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 ocf-ns_dns: 2016/02/17_18:46:11 INFO: dnsmasq daemon running Feb 17 18:46:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:28733 - exited with rc=0 Feb 17 18:46:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:28733:stderr [ -- empty -- ] Feb 17 18:46:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:28733:stdout [ -- empty -- ] Feb 17 18:46:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:28733 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:11 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:46:12 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:46:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:46:13 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff Feb 17 18:46:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(28746) = 0: Resource temporarily unavailable (11) Feb 17 18:46:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(28771) = 0: Resource temporarily unavailable (11) Feb 17 18:46:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:28808 - exited with rc=8 Feb 17 18:46:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:28808:stderr [ -- empty -- ] Feb 17 18:46:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:28808:stdout [ vrouter ] Feb 17 18:46:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:28808:stdout [ mgmt-conntrd ] Feb 17 18:46:13 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:28808 exit-code:8 exec-time:0ms queue-time:0ms Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(28746) = 0: Resource temporarily unavailable (11) Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(28771) = 0: Resource temporarily unavailable (11) Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:28833 - exited with rc=0 Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:28833:stderr [ -- empty -- ] Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:28833:stdout [ -- empty -- ] Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:28833 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:14 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:14 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:14 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 27fd77296cbdb451c172ddb150a2aa38 for 0.155.683 (0x1733cf0 0) Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:14 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:14 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:14 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:14 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:46:14 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(28746) = 0: Resource temporarily unavailable (11) Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(28771) = 0: Resource temporarily unavailable (11) Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:28865 - exited with rc=0 Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:28865:stderr [ -- empty -- ] Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:28865:stdout [ -- empty -- ] Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:28865 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of ping_vip__public_monitor_20000 ocf-ns_IPaddr2: 2016/02/17_18:46:14 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.687ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.712ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.544ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.538ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:28746 - exited with rc=0 Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:28746:stderr [ -- empty -- ] Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:28746:stdout [ -- empty -- ] Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:28746 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(28771) = 0: Resource temporarily unavailable (11) Feb 17 18:46:14 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(28901) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:46:15 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.848ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.534ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.537ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:46:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:28771 - exited with rc=0 Feb 17 18:46:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:28771:stderr [ -- empty -- ] Feb 17 18:46:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:28771:stdout [ -- empty -- ] Feb 17 18:46:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:28771 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:15 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(28901) = 0: Resource temporarily unavailable (11) Feb 17 18:46:16 [28951] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:16 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=28951 id=faf7597d-4ad2-4bda-ba08-4bef9fc382e6 Feb 17 18:46:16 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-28951-11) Feb 17 18:46:16 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [28951] Feb 17 18:46:16 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:16 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:16 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:16 [28951] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:16 [28951] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:16 [28951] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:16 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: pingd=1000 Feb 17 18:46:16 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1000, Current: 1000, Stored: 1000 Feb 17 18:46:16 [28951] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: pingd=1000 for localhost Feb 17 18:46:16 [28951] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:28901 - exited with rc=0 Feb 17 18:46:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:28901:stderr [ -- empty -- ] Feb 17 18:46:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:28901:stdout [ -- empty -- ] Feb 17 18:46:16 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:ping_vip__public action:monitor call_id:140 pid:28901 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:16 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-28951-11) Feb 17 18:46:16 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-28951-11) state:2 Feb 17 18:46:16 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:16 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-28951-11-header Feb 17 18:46:16 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-28951-11-header Feb 17 18:46:16 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-28951-11-header Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:17 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:17 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:17 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:17 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:17 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:46:17 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:46:17 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:17 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:17 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:17 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:17 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:17 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:46:17 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:46:17 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:46:17 [6707] node-3.test.domain.local crmd: debug: throttle_update: Host node-2.test.domain.local supports a maximum of 4 jobs and throttle mode 0000. New job limit is 4 Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:46:17 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:46:19 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:46:20 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:46:22 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.621ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.640ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.544ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.532ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:46:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:28985 - exited with rc=0 Feb 17 18:46:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:28985:stderr [ -- empty -- ] Feb 17 18:46:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:28985:stdout [ -- empty -- ] Feb 17 18:46:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:28985 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:22 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(29015) = 0: Resource temporarily unavailable (11) Feb 17 18:46:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_rabbitmq-server_monitor_30000 Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-29104-26) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [29104] Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761349cf10, cpd=0x7f761349d7f4 Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f761349cf10 Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-29104-26) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-29104-26) state:2 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-6268-29104-26-header Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-6268-29104-26-header Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-6268-29104-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-29104-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-29104-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-29104-26-header Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.001667 (5 ticks in 30s) Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 1.400000 (full: 1.40 0.84 0.52 4/523 29104) Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: throttle_timer_cb: New throttle mode: 0000 (was 0001) Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: info: throttle_send_command: Updated throttle state to 0000 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-29104-26) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [29104] Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-29104-26-header Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-29104-26-header Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-29104-26-header Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:46:23 [29104] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-29104-26) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-29104-26) state:2 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-29104-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-29104-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-29104-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6707-26) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6707] Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:46:23 [6707] node-3.test.domain.local crmd: debug: throttle_update: Host node-3.test.domain.local supports a maximum of 4 jobs and throttle mode 0000. New job limit is 4 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6707-26) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-26) state:2 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6707-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6707-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6707-26-header ocf-ns_IPaddr2: 2016/02/17_18:46:23 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.879ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.539ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.544ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:46:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:29015 - exited with rc=0 Feb 17 18:46:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:29015:stderr [ -- empty -- ] Feb 17 18:46:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:29015:stdout [ -- empty -- ] Feb 17 18:46:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:29015 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(29088) = 0: Resource temporarily unavailable (11) lrmd: 2016/02/17_18:46:23 DEBUG: p_rabbitmq-server: monitor: action start. lrmd: 2016/02/17_18:46:23 INFO: p_rabbitmq-server: get_monitor(): CHECK LEVEL IS: 0 lrmd: 2016/02/17_18:46:23 INFO: p_rabbitmq-server: get_monitor(): get_status() returns 0. lrmd: 2016/02/17_18:46:23 INFO: p_rabbitmq-server: get_monitor(): also checking if we are master. Feb 17 18:46:23 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:46:23 [29385] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=29385 id=67508258-eaa3-4994-aa02-ae7e7ba2a3fc Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-29385-11) Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [29385] Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29385] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29385] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29385] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:46:23 [29385] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:46:23 [29385] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-29385-11) Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-29385-11) state:2 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-29385-11-header Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-29385-11-header Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-29385-11-header Feb 17 18:46:23 [29388] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=29388 id=44bbd6dd-fb68-4254-ae9d-4a4586410de9 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-29388-11) Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [29388] Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29388] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29388] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29388] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:46:23 [29388] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:46:23 [29388] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-29388-11) Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-29388-11) state:2 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-29388-11-header Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-29388-11-header Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-29388-11-header Feb 17 18:46:23 [29403] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=29403 id=a5476f28-10d2-473b-a790-5232386d6778 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-29403-11) Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [29403] Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29403] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29403] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29403] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=2850 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2850, Current: 2850, Stored: 2850 Feb 17 18:46:23 [29403] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=2850 for localhost Feb 17 18:46:23 [29403] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-29403-11) Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-29403-11) state:2 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-29403-11-header Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-29403-11-header Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-29403-11-header Feb 17 18:46:23 [29411] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=29411 id=aa79965b-88a8-4e9c-9340-90c0d7d0de59 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-29411-11) Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [29411] Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29411] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29411] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29411] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:46:23 [29411] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:46:23 [29411] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-29411-11) Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-29411-11) state:2 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-29411-11-header Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-29411-11-header Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-29411-11-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:23 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.683 2 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.684 (null) Feb 17 18:46:23 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=684 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.47 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1021, version=0.155.684) Feb 17 18:46:23 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.684 to 0.155.683 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:23 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/321) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:23 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/321, version=0.155.684) Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1439bb0 for uid=0 gid=0 pid=29417 id=4f95f8ef-abcf-4b54-9abc-ed82d2c46a58 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-29417-14) Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [29417] Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 321 for cpu_load=0.87, passed Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit-master'] does not exist Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: info: main: Read rabbit-master= Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-29417-14-header Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-29417-14) Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-29417-14) state:2 Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-29417-14-header Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-29417-14-header Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-29417-14-header Feb 17 18:46:23 [29417] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-29417-14-header Feb 17 18:46:23 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-29417-14-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:23 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:23 [29413] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining lrmd: 2016/02/17_18:46:23 INFO: p_rabbitmq-server: get_monitor(): master attribute is 1 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=29413 id=243a122d-c1ac-4b82-b386-b78faf19b725 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-29413-11) Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [29413] Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29413] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29413] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29413] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:46:23 [29413] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:46:23 [29413] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-29413-11) Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-29413-11) state:2 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-29413-11-header Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-29413-11-header Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-29413-11-header Feb 17 18:46:23 [29433] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=29433 id=4859b9e5-834c-4d8e-81ba-4072d025edea Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-29433-11) Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [29433] Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29433] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29433] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [29433] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:46:23 [29433] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:46:23 [29433] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-29433-11) Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-29433-11) state:2 Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-29433-11-header Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-29433-11-header Feb 17 18:46:23 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-29433-11-header Feb 17 18:46:24 [29441] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=29441 id=5bb8c07c-0772-416f-9020-5315459e86c0 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-29441-11) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [29441] Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29441] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29441] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29441] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.84, Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.84,, Current: 0.87,, Stored: 0.87, Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.84, Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.84,) Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:46:24 [29441] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.84, for localhost Feb 17 18:46:24 [29441] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:24 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/323) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 323: cpu_load=0.84, Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-29441-11) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-29441-11) state:2 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-29441-11-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-29441-11-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-29441-11-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df80 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.684 2 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.685 (null) Feb 17 18:46:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=685 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.84, Feb 17 18:46:24 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/323, version=0.155.685) Feb 17 18:46:24 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.685 to 0.155.684 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 323 for cpu_load=0.84, passed Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df80 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:46:24 [29449] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=29449 id=93cef67b-f707-41b5-b869-b8ee34069352 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-29449-11) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [29449] Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29449] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29449] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29449] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:46:24 [29449] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:46:24 [29449] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-29449-11) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-29449-11) state:2 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-29449-11-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-29449-11-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:24 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1023, version=0.155.685) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-29449-11-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df20 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.685 2 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.686 (null) Feb 17 18:46:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=686 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9056 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1025, version=0.155.686) Feb 17 18:46:24 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.686 to 0.155.685 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:24 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/325) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df20 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:24 [29454] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:24 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/325, version=0.155.686) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:24 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=29454 id=21675df9-545f-428a-a21b-5ba6baa610cd Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-29454-11) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [29454] Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 325 for var_log_free=9056 passed Feb 17 18:46:24 [29454] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29454] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29454] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=600 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 600, Current: 600, Stored: 600 Feb 17 18:46:24 [29454] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=600 for localhost Feb 17 18:46:24 [29454] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-29454-11) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-29454-11) state:2 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-29454-11-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-29454-11-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-29454-11-header Feb 17 18:46:24 [29519] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=29519 id=ba54c53d-966e-4773-915f-bb729573bbe7 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-29519-11) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [29519] Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29519] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29519] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29519] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11691 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11691, Current: 11691, Stored: 11691 Feb 17 18:46:24 [29519] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11691 for localhost Feb 17 18:46:24 [29519] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-29519-11) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-29519-11) state:2 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-29519-11-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-29519-11-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-29519-11-header Feb 17 18:46:24 [29532] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=29532 id=0cf2d74c-0cee-4d7d-9740-1ded4814a70a Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-29532-11) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [29532] Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29532] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29532] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29532] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11691 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11691, Current: 11691, Stored: 11691 Feb 17 18:46:24 [29532] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11691 for localhost Feb 17 18:46:24 [29532] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-29532-11) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-29532-11) state:2 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-29532-11-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-29532-11-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-29532-11-header Feb 17 18:46:24 [29548] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=29548 id=9988ac69-612c-48a2-ba83-0d27923986d3 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-29548-11) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [29548] Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29548] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29548] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29548] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9056 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9056, Current: 9056, Stored: 9056 Feb 17 18:46:24 [29548] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9056 for localhost Feb 17 18:46:24 [29548] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-29548-11) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-29548-11) state:2 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-29548-11-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-29548-11-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-29548-11-header Feb 17 18:46:24 [29560] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=29560 id=8fb1d8ec-cb92-41a8-ba68-960baa572724 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-29560-11) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [29560] Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29560] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29560] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29560] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18422 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18422, Current: 18422, Stored: 18422 Feb 17 18:46:24 [29560] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18422 for localhost Feb 17 18:46:24 [29560] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-29560-11) Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-29560-11) state:2 Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-29560-11-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-29560-11-header Feb 17 18:46:24 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-29560-11-header Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(29088) = 0: Resource temporarily unavailable (11) Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:29377 - exited with rc=0 Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:29377:stderr [ -- empty -- ] Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:29377:stdout [ arch: x86_64 ] Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:29377:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:29377:stdout [ free_swap: 2850 ] Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:29377:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:29377:stdout [ cpu_speed: 6999.99 ] Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:29377:stdout [ cpu_cores: 2 ] Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:29377:stdout [ cpu_load: 0.84, ] Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:29377:stdout [ ram_total: 3050 ] Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:29377:stdout [ ram_free: 600 ] Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:29377:stdout [ root_free: 11691 ] Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:29377:stdout [ root_free: 11691 ] Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:29377:stdout [ var_log_free: 9056 ] Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:29377:stdout [ var_lib_mysql_free: 18422 ] Feb 17 18:46:24 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:29377 exit-code:0 exec-time:0ms queue-time:0ms lrmd: 2016/02/17_18:46:24 INFO: p_rabbitmq-server: get_monitor(): checking if rabbit app is running lrmd: 2016/02/17_18:46:24 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running. checking if we are the part of healthy cluster Feb 17 18:46:24 [29618] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:46:24 [29618] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:46:24 [29618] node-3.test.domain.local crm_node: debug: main: Attempting to process -p command for cluster type: corosync Feb 17 18:46:24 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=29618 id=a9ae9a61-1643-4e88-a33e-71d45a33edc6 Feb 17 18:46:24 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-29618-12) Feb 17 18:46:24 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [29618] Feb 17 18:46:24 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29618] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29618] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29618] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:24 [29618] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:24 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-29618-12) Feb 17 18:46:24 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-29618-12) state:2 Feb 17 18:46:24 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:24 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-29618-12-header Feb 17 18:46:24 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-29618-12-header Feb 17 18:46:24 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-29618-12-header lrmd: 2016/02/17_18:46:24 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running. looking for master on node-2.test.domain.local Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1439bb0 for uid=0 gid=0 pid=29625 id=1516ed24-70d1-4e63-9b1c-a3b1eb708040 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-29625-14) Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [29625] Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-2.test.domain.local to 2 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='2']//transient_attributes//nvpair[@name='rabbit-master'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[25]) Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: info: main: Read rabbit-master=true Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-29625-14) Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-29625-14) state:2 Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-29625-14-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-29625-14-header Feb 17 18:46:24 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-29625-14-header Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-29625-14-header Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-29625-14-header Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-29625-14-header Feb 17 18:46:24 [29625] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 lrmd: 2016/02/17_18:46:24 INFO: p_rabbitmq-server: get_monitor(): fetched master attribute for node-2.test.domain.local. attr value is 0 lrmd: 2016/02/17_18:46:24 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running. master is node-2.test.domain.local lrmd: 2016/02/17_18:46:25 INFO: p_rabbitmq-server: get_monitor(): rabbit app is running and is member of healthy cluster lrmd: 2016/02/17_18:46:25 INFO: p_rabbitmq-server: get_monitor(): preparing to update master score for node Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1439bb0 for uid=0 gid=0 pid=29749 id=e020ef77-401b-4635-8a7a-d83d9758cdaa Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-29749-14) Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [29749] Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit-start-time'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[23]) Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: info: main: Read rabbit-start-time=1455734428 Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-29749-14) Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-29749-14) state:2 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-29749-14-header Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-29749-14-header Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-29749-14-header Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-29749-14-header Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-29749-14-header Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-29749-14-header Feb 17 18:46:25 [29749] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:25 [29757] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:46:25 [29757] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:46:25 [29757] node-3.test.domain.local crm_node: debug: main: Attempting to process -p command for cluster type: corosync Feb 17 18:46:25 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x1cff240 for uid=0 gid=0 pid=29757 id=2131909a-e232-47a5-beaa-37757c456009 Feb 17 18:46:25 [6556] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6556-29757-12) Feb 17 18:46:25 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [29757] Feb 17 18:46:25 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [29757] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [29757] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [29757] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [29757] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:25 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6556-29757-12) Feb 17 18:46:25 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6556-29757-12) state:2 Feb 17 18:46:25 [6556] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:25 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6556-29757-12-header Feb 17 18:46:25 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6556-29757-12-header Feb 17 18:46:25 [6556] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6556-29757-12-header Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1439bb0 for uid=0 gid=0 pid=29761 id=bf0e1fc4-b271-4f00-a317-449b798dd3bb Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-29761-14) Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [29761] Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-2.test.domain.local to 2 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='2']//transient_attributes//nvpair[@name='rabbit-start-time'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[26]) Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: info: main: Read rabbit-start-time=1455734419 Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-29761-14-header Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-29761-14-header Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-29761-14-header Feb 17 18:46:25 [29761] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-29761-14) Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-29761-14) state:2 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-29761-14-header Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-29761-14-header Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-29761-14-header lrmd: 2016/02/17_18:46:25 INFO: p_rabbitmq-server: get_monitor(): comparing our uptime (357) with node-2.test.domain.local (366) lrmd: 2016/02/17_18:46:25 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl list_channels 2>&1 > /dev/null lrmd: 2016/02/17_18:46:25 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl list_channels 2>&1 > /dev/null Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1439bb0 for uid=0 gid=0 pid=29869 id=fae78a03-678c-4fe2-9075-819967bdf689 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-29869-14) Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [29869] Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=29869 id=8ecd7771-d201-4dc8-8b3a-4c76eced3eb8 Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-29869-11) Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [29869] Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_list_channels_timeouts=0 Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_list_channels_timeouts=0 for node-3.test.domain.local Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: info: main: Update rabbit_list_channels_timeouts=0 sent via attrd Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-29869-14) Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-29869-14) state:2 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-29869-14-header Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-29869-14-header Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-29869-14-header Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-29869-14-header Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-29869-14-header Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-29869-14-header Feb 17 18:46:25 [29869] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: 0, Stored: 0 Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-29869-11) Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-29869-11) state:2 Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-29869-11-header Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-29869-11-header Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-29869-11-header lrmd: 2016/02/17_18:46:25 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl -q eval 'rabbit_alarm:get_alarms().' lrmd: 2016/02/17_18:46:25 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl -q eval 'rabbit_alarm:get_alarms().' Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1439bb0 for uid=0 gid=0 pid=29973 id=e1ee63d8-46f6-4b63-b972-d9d471e1a9f7 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-29973-14) Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [29973] Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=29973 id=f8dad66b-d0c1-4f1e-95d2-d99af9959d23 Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-29973-11) Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [29973] Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_get_alarms_timeouts=0 for node-3.test.domain.local Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: info: main: Update rabbit_get_alarms_timeouts=0 sent via attrd Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-29973-14-header Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-29973-14-header Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-29973-14-header Feb 17 18:46:25 [29973] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-29973-14) Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-29973-14) state:2 Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-29973-14-header Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-29973-14-header Feb 17 18:46:25 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-29973-14-header Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_get_alarms_timeouts=0 Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: 0, Stored: 0 Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-29973-11) Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-29973-11) state:2 Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-29973-11-header Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-29973-11-header Feb 17 18:46:25 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-29973-11-header lrmd: 2016/02/17_18:46:25 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl -q list_queues memory messages consumer_utilisation lrmd: 2016/02/17_18:46:26 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl -q list_queues memory messages consumer_utilisation Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1439bb0 for uid=0 gid=0 pid=30105 id=8df6d3ec-c7f1-4b8f-9374-d3b60a4ec019 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-30105-14) Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [30105] Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30105 id=97f9766a-09fa-48e9-8da7-a42a92df2932 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30105-11) Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30105] Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_list_queues_timeouts=0 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: 0, Stored: 0 Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_list_queues_timeouts=0 for node-3.test.domain.local Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: info: main: Update rabbit_list_queues_timeouts=0 sent via attrd Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-30105-14-header Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-30105-14-header Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-30105-14-header Feb 17 18:46:26 [30105] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30105-11) Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30105-11) state:2 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30105-11-header Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30105-11-header Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30105-11-header Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-30105-14) Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-30105-14) state:2 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-30105-14-header Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-30105-14-header Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-30105-14-header lrmd: 2016/02/17_18:46:26 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl -q status Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_mysql_monitor_60000 ocf-mysql-wss: 2016/02/17_18:46:26 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:46:26 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:46:26 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:46:26 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:46:26 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:21543 ocf-mysql-wss: 2016/02/17_18:46:26 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:21543 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1439bb0 for uid=0 gid=0 pid=30270 id=2d8e4678-b455-472b-8674-c4f6b3c542c1 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6655-30270-14) Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [30270] Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30270 id=603e92c1-4401-4d44-8a23-2f536d8d82ed Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30270-11) Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30270] Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21543 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 71dae69f-d579-11e5-a980-12eca8fb60f4:21543, Current: (null), Stored: (null) Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of gtid is 71dae69f-d579-11e5-a980-12eca8fb60f4:21543 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: gtid (71dae69f-d579-11e5-a980-12eca8fb60f4:21543) Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] does not exist Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21543 for node-3.test.domain.local Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: info: main: Update gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21543 sent via attrd Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6655-30270-14-header Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6655-30270-14-header Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6655-30270-14-header Feb 17 18:46:26 [30270] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6655-30270-14) Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-30270-14) state:2 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-30270-14-header Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-30270-14-header Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-30270-14-header Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 327: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21543 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30270-11) Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30270-11) state:2 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30270-11-header Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30270-11-header Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30270-11-header Feb 17 18:46:26 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/327) Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349df60 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.686 2 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.687 (null) Feb 17 18:46:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=687 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:46:26 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/327, version=0.155.687) Feb 17 18:46:26 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.687 to 0.155.686 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:26 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 327 for gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21543 passed Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349df60 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:26 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:26 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1027, version=0.155.687) Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:26 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-mysql-wss: 2016/02/17_18:46:26 DEBUG: MySQL monitor succeeded Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(29088) = 0: Resource temporarily unavailable (11) Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_60000:30189 - exited with rc=0 Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_monitor_60000:30189:stderr [ Warning: Using a password on the command line interface can be insecure. ] Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_monitor_60000:30189:stderr [ Warning: Using a password on the command line interface can be insecure. ] Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_monitor_60000:30189:stderr [ Warning: Using a password on the command line interface can be insecure. ] Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_monitor_60000:30189:stderr [ Warning: Using a password on the command line interface can be insecure. ] Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_monitor_60000:30189:stderr [ Warning: Using a password on the command line interface can be insecure. ] Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_60000:30189:stdout [ -- empty -- ] Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_mysql action:monitor call_id:143 pid:30189 exit-code:0 exec-time:0ms queue-time:0ms lrmd: 2016/02/17_18:46:26 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl -q status lrmd: 2016/02/17_18:46:26 INFO: p_rabbitmq-server: get_monitor(): RabbitMQ is running 201 queues consuming 7.17377m of 6081m total, with 0 queued messages, average consumer utilization 1 lrmd: 2016/02/17_18:46:26 INFO: p_rabbitmq-server: get_monitor(): RabbitMQ status: [{pid,12415}, {running_applications, [{rabbitmq_management,"RabbitMQ Management Console","3.5.6"}, {rabbitmq_management_agent,"RabbitMQ Management Agent","3.5.6"}, {rabbit,"RabbitMQ","3.5.6"}, {os_mon,"CPO CXC 138 46","2.4"}, {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.5.6"}, {webmachine,"webmachine","1.10.3-rmq3.5.6-gite9359c7"}, {mochiweb,"MochiMedia Web Server","2.7.0-rmq3.5.6-git680dba8"}, {amqp_client,"RabbitMQ AMQP Client","3.5.6"}, {mnesia,"MNESIA CXC 138 12","4.13.1"}, {xmerl,"XML parser","1.3.8"}, {inets,"INETS CXC 138 49","6.0.1"}, {sasl,"SASL CXC 138 11","2.6"}, {stdlib,"ERTS CXC 138 10","2.6"}, {kernel,"ERTS CXC 138 10","4.1"}]}, {os,{unix,linux}}, {erlang_version, "Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:2:2] [async-threads:4] [kernel-poll:true]\n"}, {memory, [{total,113155592}, {connection_readers,390096}, {connection_writers,118360}, {connection_channels,861208}, {connection_other,851432}, {queue_procs,3282728}, {queue_slave_procs,2415600}, {plugins,86776}, {other_proc,13533928}, {mnesia,912032}, {mgmt_db,11528}, {msg_index,141056}, {other_ets,1368896}, {binary,54487616}, {code,20003841}, {atom,711569}, {other_system,13978926}]}, {alarms,[]}, {listeners,[{clustering,41055,"::"},{amqp,5673,"192.168.0.4"}]}, {vm_memory_high_watermark,0.4}, {vm_memory_limit,1262442905}, {disk_free_limit,5000000}, {disk_free,12258816000}, {file_descriptors, [{total_limit,102300}, {total_used,46}, {sockets_limit,92068}, {sockets_used,44}]}, {processes,[{limit,1048576},{used,1209}]}, {run_queue,0}, {uptime,697}] lrmd: 2016/02/17_18:46:26 INFO: p_rabbitmq-server: get_monitor(): get_monitor function ready to return 0 lrmd: 2016/02/17_18:46:26 DEBUG: p_rabbitmq-server: monitor: role: lrmd: 2016/02/17_18:46:26 DEBUG: p_rabbitmq-server: monitor: result: 0 lrmd: 2016/02/17_18:46:27 DEBUG: p_rabbitmq-server: monitor: action end. Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_monitor_30000:29088 - exited with rc=0 Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:29088:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:29088:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:29088:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:29088:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:29088:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:29088:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_monitor_30000:29088:stdout [ -- empty -- ] Feb 17 18:46:26 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_rabbitmq-server action:monitor call_id:139 pid:29088 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:27 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:46:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:46:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:46:28 INFO: ntp daemon running Feb 17 18:46:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(30316) = 0: Resource temporarily unavailable (11) Feb 17 18:46:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(30352) = 0: Resource temporarily unavailable (11) Feb 17 18:46:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:30378 - exited with rc=0 Feb 17 18:46:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:30378:stderr [ -- empty -- ] Feb 17 18:46:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:30378:stdout [ -- empty -- ] Feb 17 18:46:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:30378 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:46:28 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:46:28 INFO: net.ipv4.ip_forward = 1 Feb 17 18:46:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(30316) = 0: Resource temporarily unavailable (11) Feb 17 18:46:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(30352) = 0: Resource temporarily unavailable (11) Feb 17 18:46:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:30389 - exited with rc=0 Feb 17 18:46:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:30389:stderr [ -- empty -- ] Feb 17 18:46:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:30389:stdout [ -- empty -- ] Feb 17 18:46:28 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 pid:30389 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:46:30 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.617ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.642ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.540ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.545ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:30316 - exited with rc=0 Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:30316:stderr [ -- empty -- ] Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:30316:stdout [ -- empty -- ] Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:30316 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(30352) = 0: Resource temporarily unavailable (11) Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:46:30 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(30352) = 0: Resource temporarily unavailable (11) Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:30439 - exited with rc=0 Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:30439:stderr [ -- empty -- ] Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:30439:stdout [ -- empty -- ] Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:30439 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(30352) = 0: Resource temporarily unavailable (11) Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:30466 - exited with rc=0 Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:30466:stderr [ -- empty -- ] Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:30466:stdout [ -- empty -- ] Feb 17 18:46:30 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:30466 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 ocf-ns_dns: 2016/02/17_18:46:31 INFO: dnsmasq daemon running Feb 17 18:46:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(30352) = 0: Resource temporarily unavailable (11) Feb 17 18:46:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:30491 - exited with rc=0 Feb 17 18:46:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:30491:stderr [ -- empty -- ] Feb 17 18:46:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:30491:stdout [ -- empty -- ] Feb 17 18:46:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:30491 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:46:31 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.811ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.541ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.529ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:46:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:30352 - exited with rc=0 Feb 17 18:46:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:30352:stderr [ -- empty -- ] Feb 17 18:46:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:30352:stdout [ -- empty -- ] Feb 17 18:46:31 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:30352 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:31 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:31 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:31 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:31 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:31 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:31 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:31 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:31 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:31 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 9ecd43e8496e7a884b65574e83359d1d for 0.155.687 (0x1733cf0 0) Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:31 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:31 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:31 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:31 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:31 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:31 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:31 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:31 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:46:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:30540 - exited with rc=0 Feb 17 18:46:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:30540:stderr [ -- empty -- ] Feb 17 18:46:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:30540:stdout [ -- empty -- ] Feb 17 18:46:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:30540 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 ocf-neutron-dhcp-agent: 2016/02/17_18:46:34 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:46:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:30573 - exited with rc=0 Feb 17 18:46:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:30573:stderr [ -- empty -- ] Feb 17 18:46:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:30573:stdout [ -- empty -- ] Feb 17 18:46:34 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:30573 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:46:35 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:46:35 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:46:35 INFO: haproxy daemon running Feb 17 18:46:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:30613 - exited with rc=0 Feb 17 18:46:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:30613:stderr [ -- empty -- ] Feb 17 18:46:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:30613:stdout [ -- empty -- ] Feb 17 18:46:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 pid:30613 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:35 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:46:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of ping_vip__public_monitor_20000 Feb 17 18:46:36 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:46:38 [30737] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:38 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30737 id=e658917c-df43-4cb2-8e86-f378d69f3348 Feb 17 18:46:38 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30737-11) Feb 17 18:46:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30737] Feb 17 18:46:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:38 [30737] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:38 [30737] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:38 [30737] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:38 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: pingd=1000 Feb 17 18:46:38 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1000, Current: 1000, Stored: 1000 Feb 17 18:46:38 [30737] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: pingd=1000 for localhost Feb 17 18:46:38 [30737] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(30639) = 0: Resource temporarily unavailable (11) Feb 17 18:46:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:30676 - exited with rc=0 Feb 17 18:46:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:30676:stderr [ -- empty -- ] Feb 17 18:46:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:30676:stdout [ -- empty -- ] Feb 17 18:46:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:ping_vip__public action:monitor call_id:140 pid:30676 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(30689) = 0: Resource temporarily unavailable (11) Feb 17 18:46:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30737-11) Feb 17 18:46:38 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30737-11) state:2 Feb 17 18:46:38 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30737-11-header Feb 17 18:46:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30737-11-header Feb 17 18:46:38 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30737-11-header ocf-ns_IPaddr2: 2016/02/17_18:46:38 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.682ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.720ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.544ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.543ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:46:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:30639 - exited with rc=0 Feb 17 18:46:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:30639:stderr [ -- empty -- ] Feb 17 18:46:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:30639:stdout [ -- empty -- ] Feb 17 18:46:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:30639 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:38 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(30689) = 0: Resource temporarily unavailable (11) Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:46:39 [30756] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30756 id=f53bed6f-947b-49f4-a2ad-0aa0754b4f54 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30756-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30756] Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30756] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30756] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30756] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:46:39 [30756] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:46:39 [30756] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30756-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30756-11) state:2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30756-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30756-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30756-11-header Feb 17 18:46:39 [30759] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30759 id=20f6f032-a5d0-4013-ae22-6eec87d15749 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30759-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30759] Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30759] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30759] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30759] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:46:39 [30759] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:46:39 [30759] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30759-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30759-11) state:2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30759-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30759-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30759-11-header Feb 17 18:46:39 [30773] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30773 id=6d9d4f62-c35b-477e-8f68-a1fc9accfedf Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30773-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30773] Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30773] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30773] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30773] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=2850 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2850, Current: 2850, Stored: 2850 Feb 17 18:46:39 [30773] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=2850 for localhost Feb 17 18:46:39 [30773] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30773-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30773-11) state:2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30773-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30773-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30773-11-header Feb 17 18:46:39 [30780] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30780 id=0941a5aa-4d7b-4435-9c11-a4598ff559d5 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30780-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30780] Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30780] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30780] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30780] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:46:39 [30780] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:46:39 [30780] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30780-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30780-11) state:2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30780-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30780-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30780-11-header Feb 17 18:46:39 [30781] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30781 id=ada37a35-569e-45d9-af80-c3a5b769a35b Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30781-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30781] Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30781] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30781] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30781] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:46:39 [30781] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:46:39 [30781] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30781-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30781-11) state:2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30781-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30781-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30781-11-header Feb 17 18:46:39 [30782] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30782 id=32e61e39-a799-49a9-9c08-a2c464922e7b Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30782-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30782] Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30782] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30782] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30782] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:46:39 [30782] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:46:39 [30782] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30782-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30782-11) state:2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30782-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30782-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30782-11-header Feb 17 18:46:39 [30787] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30787 id=ec0940fd-8e2a-4af5-97ee-ebea16916c09 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30787-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30787] Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30787] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30787] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30787] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.85, Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.85,, Current: 0.84,, Stored: 0.84, Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.85, Feb 17 18:46:39 [30787] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.85, for localhost Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.85,) Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:46:39 [30787] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:39 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/329) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 329: cpu_load=0.85, Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dd40 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:39 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.687 2 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.688 (null) Feb 17 18:46:39 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=688 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.85, Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:39 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/329, version=0.155.688) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:39 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.688 to 0.155.687 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 329 for cpu_load=0.85, passed Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30787-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30787-11) state:2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30787-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30787-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30787-11-header Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dd40 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:39 [30792] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:39 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1029, version=0.155.688) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30792 id=2b3fb03c-fb38-46de-aedf-f4df295c8b20 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30792-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30792] Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30792] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30792] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30792] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:46:39 [30792] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:46:39 [30792] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30792-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30792-11) state:2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30792-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30792-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30792-11-header Feb 17 18:46:39 [30797] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30797 id=6ea5e642-e39c-4751-b69c-58cd4185ce33 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30797-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30797] Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30797] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30797] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30797] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=600 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 600, Current: 600, Stored: 600 Feb 17 18:46:39 [30797] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=600 for localhost Feb 17 18:46:39 [30797] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30797-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30797-11) state:2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30797-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30797-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30797-11-header Feb 17 18:46:39 [30809] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30809 id=16c613d1-9ce4-4bdd-a10c-27f02aee3cc3 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30809-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30809] Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30809] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30809] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30809] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11691 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11691, Current: 11691, Stored: 11691 Feb 17 18:46:39 [30809] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11691 for localhost Feb 17 18:46:39 [30809] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30809-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30809-11) state:2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30809-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30809-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30809-11-header Feb 17 18:46:39 [30821] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30821 id=25ba9188-0f84-4500-81be-610d97ed7358 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30821-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30821] Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30821] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30821] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30821] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11691 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11691, Current: 11691, Stored: 11691 Feb 17 18:46:39 [30821] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11691 for localhost Feb 17 18:46:39 [30821] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30821-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30821-11) state:2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30821-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30821-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30821-11-header Feb 17 18:46:39 [30833] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30833 id=dbf2673f-67ae-4786-a9dc-1187c095110a Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30833-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30833] Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30833] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30833] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30833] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9055 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9055, Current: 9056, Stored: 9056 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9055 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9055) Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 331: var_log_free=9055 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/331) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-27) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349d970 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:39 [30833] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9055 for localhost Feb 17 18:46:39 [30833] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30833-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30833-11) state:2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30833-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30833-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30833-11-header Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:39 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.688 2 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.689 (null) Feb 17 18:46:39 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=689 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9055 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/331, version=0.155.689) Feb 17 18:46:39 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.689 to 0.155.688 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 331 for var_log_free=9055 passed Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-27) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-27) state:2 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349d970 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-27-header Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-27-header Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-27-header Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:39 [30845] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=30845 id=6b528144-4efd-4329-bb70-a9cd0f058fb7 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-30845-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [30845] Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30845] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30845] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [30845] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18422 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18422, Current: 18422, Stored: 18422 Feb 17 18:46:39 [30845] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18422 for localhost Feb 17 18:46:39 [30845] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-30845-11) Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-30845-11) state:2 Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-30845-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-30845-11-header Feb 17 18:46:39 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-30845-11-header Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(30689) = 0: Resource temporarily unavailable (11) Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:30748 - exited with rc=0 Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:30748:stderr [ -- empty -- ] Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:30748:stdout [ arch: x86_64 ] Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:30748:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:30748:stdout [ free_swap: 2850 ] Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:30748:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:30748:stdout [ cpu_speed: 6999.99 ] Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:30748:stdout [ cpu_cores: 2 ] Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:30748:stdout [ cpu_load: 0.85, ] Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:30748:stdout [ ram_total: 3050 ] Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:30748:stdout [ ram_free: 600 ] Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:30748:stdout [ root_free: 11691 ] Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:30748:stdout [ root_free: 11691 ] Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:30748:stdout [ var_log_free: 9055 ] Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:30748:stdout [ var_lib_mysql_free: 18422 ] Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:30748 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:39 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:39 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1031, version=0.155.689) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:39 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-ns_IPaddr2: 2016/02/17_18:46:39 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.812ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.536ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.543ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:30689 - exited with rc=0 Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:30689:stderr [ -- empty -- ] Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:30689:stdout [ -- empty -- ] Feb 17 18:46:39 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:30689 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_conntrackd_monitor_27000 ocf-ns_conntrackd: 2016/02/17_18:46:40 INFO: 25: conntrd: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 7e:fe:ef:45:54:ea brd ff:ff:ff:ff:ff:ff Feb 17 18:46:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:30858 - exited with rc=8 Feb 17 18:46:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:30858:stderr [ -- empty -- ] Feb 17 18:46:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:30858:stdout [ vrouter ] Feb 17 18:46:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_27000:30858:stdout [ mgmt-conntrd ] Feb 17 18:46:40 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 pid:30858 exit-code:8 exec-time:0ms queue-time:0ms Feb 17 18:46:43 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:44 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:44 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:44 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:44 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:44 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:44 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:44 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:44 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:44 [6655] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: f2b707559ed0eadd71101631182011f8 for 0.155.689 (0x1733cf0 0) Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:44 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:44 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:44 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:44 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:44 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:44 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:44 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:44 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:44 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 ocf-ns_IPaddr2: 2016/02/17_18:46:46 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.648ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.674ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.559ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.540ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:46:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:30932 - exited with rc=0 Feb 17 18:46:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:30932:stderr [ -- empty -- ] Feb 17 18:46:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:30932:stdout [ -- empty -- ] Feb 17 18:46:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:30932 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:46 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(30975) = 0: Resource temporarily unavailable (11) ocf-ns_IPaddr2: 2016/02/17_18:46:47 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.736ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.525ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.545ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:46:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:30975 - exited with rc=0 Feb 17 18:46:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:30975:stderr [ -- empty -- ] Feb 17 18:46:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:30975:stdout [ -- empty -- ] Feb 17 18:46:47 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:30975 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_ntp_monitor_20000 ocf-ns_ntp: 2016/02/17_18:46:48 INFO: ntp daemon running Feb 17 18:46:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:31058 - exited with rc=0 Feb 17 18:46:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:31058:stderr [ -- empty -- ] Feb 17 18:46:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:31058:stdout [ -- empty -- ] Feb 17 18:46:48 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 pid:31058 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:46:50 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:46:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:31093 - exited with rc=0 Feb 17 18:46:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:31093:stderr [ -- empty -- ] Feb 17 18:46:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:31093:stdout [ -- empty -- ] Feb 17 18:46:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 pid:31093 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:46:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:31120 - exited with rc=0 Feb 17 18:46:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:31120:stderr [ -- empty -- ] Feb 17 18:46:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:31120:stdout [ -- empty -- ] Feb 17 18:46:50 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 pid:31120 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 ocf-ns_dns: 2016/02/17_18:46:51 INFO: dnsmasq daemon running Feb 17 18:46:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:31145 - exited with rc=0 Feb 17 18:46:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:31145:stderr [ -- empty -- ] Feb 17 18:46:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:31145:stdout [ -- empty -- ] Feb 17 18:46:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 pid:31145 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:51 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_monitor_5000 Feb 17 18:46:52 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__vrouter_pub_monitor_5000 Feb 17 18:46:53 [6707] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.002000 (6 ticks in 30s) Feb 17 18:46:53 [6707] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 1.070000 (full: 1.07 0.82 0.52 2/521 31221) Feb 17 18:46:53 [6707] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(31158) = 0: Resource temporarily unavailable (11) Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(31201) = 0: Resource temporarily unavailable (11) Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:31243 - exited with rc=0 Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:31243:stderr [ -- empty -- ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:31243:stdout [ -- empty -- ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 pid:31243 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-dhcp-agent_monitor_20000 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-27) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349dd20 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.689 2 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.690 (null) Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=690 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-free_swap']: @value=2650 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1033, version=0.155.690) Feb 17 18:46:54 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.690 to 0.155.689 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='free_swap'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[12]) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/333) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-27) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-27) state:2 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349dd20 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-27-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-27-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-27-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/333, version=0.155.690) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 333 for free_swap=2850 passed Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header ocf-neutron-dhcp-agent: 2016/02/17_18:46:54 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(31158) = 0: Resource temporarily unavailable (11) Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(31201) = 0: Resource temporarily unavailable (11) Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:31277 - exited with rc=0 Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:31277:stderr [ -- empty -- ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:31277:stdout [ -- empty -- ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 pid:31277 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.690 2 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.691 (null) Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=691 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.46 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1035, version=0.155.691) Feb 17 18:46:54 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.691 to 0.155.690 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/335) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/335, version=0.155.691) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 335 for cpu_load=0.85, passed Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:46:54 [31320] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=31320 id=508d467f-ccc9-4fe3-96e8-0be22c4778aa Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-31320-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [31320] Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31320] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31320] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31320] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:46:54 [31320] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:46:54 [31320] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-31320-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-31320-11) state:2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-31320-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-31320-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-31320-11-header Feb 17 18:46:54 [31323] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=31323 id=b640c242-ba11-43b9-ae7a-a700846eed2d Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-31323-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [31323] Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31323] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31323] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31323] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31323] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:46:54 [31323] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-31323-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-31323-11) state:2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-31323-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-31323-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-31323-11-header Feb 17 18:46:54 [31337] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=31337 id=8fbf9696-0878-4a3d-8a65-92805c94e8c6 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-31337-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [31337] Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31337] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31337] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31337] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=2850 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2850, Current: 2850, Stored: 2850 Feb 17 18:46:54 [31337] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=2850 for localhost Feb 17 18:46:54 [31337] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-31337-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-31337-11) state:2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-31337-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-31337-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-31337-11-header Feb 17 18:46:54 [31346] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=31346 id=e3e8dab6-1b0c-43c8-a122-2ed1da3103b0 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-31346-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [31346] Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31346] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31346] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31346] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:46:54 [31346] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:46:54 [31346] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-31346-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-31346-11) state:2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-31346-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-31346-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-31346-11-header Feb 17 18:46:54 [31348] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=31348 id=5bfa3aaf-81e7-41ec-a591-184c6bc06d96 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-31348-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [31348] Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31348] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31348] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31348] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:46:54 [31348] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:46:54 [31348] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-31348-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-31348-11) state:2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-31348-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-31348-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-31348-11-header Feb 17 18:46:54 [31350] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=31350 id=77cf0c38-db4f-4123-8dc1-0c343d66faab Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-31350-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [31350] Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31350] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31350] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31350] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31350] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:46:54 [31350] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-31350-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-31350-11) state:2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-31350-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-31350-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-31350-11-header Feb 17 18:46:54 [31355] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=31355 id=3f64d5f2-93b5-4673-a291-9b73c03bbf8c Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-31355-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [31355] Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31355] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31355] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31355] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.82, Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.82,, Current: 0.85,, Stored: 0.85, Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.82, Feb 17 18:46:54 [31355] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.82, for localhost Feb 17 18:46:54 [31355] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.82,) Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/337) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 337: cpu_load=0.82, Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-31355-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-31355-11) state:2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-31355-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-31355-11-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-31355-11-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.691 2 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.692 (null) Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=692 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.82, Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/337, version=0.155.692) Feb 17 18:46:54 [6656] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.692 to 0.155.691 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:54 [31360] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 337 for cpu_load=0.82, passed Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6678-26) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6678] Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=31360 id=b33c99af-cb8e-4452-80fc-2f290c8a6b5c Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-31360-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [31360] Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31360] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6678-26) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-26) state:2 Feb 17 18:46:54 [31360] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:54 [31360] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6678-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6678-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6678-26-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:46:54 [31360] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:46:54 [31360] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-31360-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-31360-11) state:2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-31360-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-31360-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-31360-11-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-6655-26) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6655] Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:54 [6655] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:46:54 [6655] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1037, version=0.155.692) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-6655-26) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-26) state:2 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-6655-26-header Feb 17 18:46:54 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-6655-26-header Feb 17 18:46:54 [31365] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=31365 id=35f71fe0-00b0-45f3-85b9-5ccbee01a44c Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-31365-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [31365] Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31365] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31365] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31365] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=600 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 600, Current: 600, Stored: 600 Feb 17 18:46:54 [31365] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=600 for localhost Feb 17 18:46:54 [31365] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-31365-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-31365-11) state:2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-31365-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-31365-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-31365-11-header Feb 17 18:46:54 [31377] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=31377 id=6ee3452b-c580-4f22-8b35-e5b259939d15 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-31377-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [31377] Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31377] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31377] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31377] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11691 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11691, Current: 11691, Stored: 11691 Feb 17 18:46:54 [31377] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11691 for localhost Feb 17 18:46:54 [31377] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-31377-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-31377-11) state:2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-31377-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-31377-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-31377-11-header Feb 17 18:46:54 [31389] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=31389 id=e6a961ea-935f-4662-9ab6-c2281f1bf1a0 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-31389-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [31389] Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31389] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31389] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31389] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11691 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11691, Current: 11691, Stored: 11691 Feb 17 18:46:54 [31389] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11691 for localhost Feb 17 18:46:54 [31389] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-31389-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-31389-11) state:2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-31389-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-31389-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-31389-11-header Feb 17 18:46:54 [31401] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=31401 id=7d30f0b5-7057-4486-98f0-a27c6937f4d4 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-31401-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [31401] Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31401] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31401] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31401] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9055 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9055, Current: 9055, Stored: 9055 Feb 17 18:46:54 [31401] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9055 for localhost Feb 17 18:46:54 [31401] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-31401-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-31401-11) state:2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-31401-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-31401-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-31401-11-header Feb 17 18:46:54 [31413] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x201bbd0 for uid=0 gid=0 pid=31413 id=c8da493c-474c-4a8f-8267-160b90474e50 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6678-31413-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [31413] Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31413] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31413] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [31413] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18422 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18422, Current: 18422, Stored: 18422 Feb 17 18:46:54 [31413] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18422 for localhost Feb 17 18:46:54 [31413] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6678-31413-11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6678-31413-11) state:2 Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-31413-11-header Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-31413-11-header Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(31158) = 0: Resource temporarily unavailable (11) Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(31201) = 0: Resource temporarily unavailable (11) Feb 17 18:46:54 [6678] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-31413-11-header Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:31311 - exited with rc=0 Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:31311:stderr [ -- empty -- ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:31311:stdout [ arch: x86_64 ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:31311:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:31311:stdout [ free_swap: 2850 ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:31311:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:31311:stdout [ cpu_speed: 6999.99 ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:31311:stdout [ cpu_cores: 2 ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:31311:stdout [ cpu_load: 0.82, ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:31311:stdout [ ram_total: 3050 ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:31311:stdout [ ram_free: 600 ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:31311:stdout [ root_free: 11691 ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:31311:stdout [ root_free: 11691 ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:31311:stdout [ var_log_free: 9055 ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:31311:stdout [ var_lib_mysql_free: 18422 ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 pid:31311 exit-code:0 exec-time:0ms queue-time:0ms ocf-ns_IPaddr2: 2016/02/17_18:46:54 INFO: ARPING 192.168.0.1 from 192.168.0.4 br-mgmt Unicast reply from 192.168.0.1 [B6:0B:8C:A8:EF:DA] 0.616ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.664ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.531ms Unicast reply from 192.168.0.1 [7E:FE:EF:45:54:EA] 0.529ms Sent 3 probes (1 broadcast(s)) Received 4 response(s) Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:31158 - exited with rc=0 Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:31158:stderr [ -- empty -- ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_monitor_5000:31158:stdout [ -- empty -- ] Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 pid:31158 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:54 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(31201) = 0: Resource temporarily unavailable (11) Feb 17 18:46:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_rabbitmq-server_monitor_103000 Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-31437-26) Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [31437] Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761349cf10, cpd=0x7f76134a6b04 Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f761349cf10 Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-6268-31437-26-header Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-6268-31437-26-header Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-6268-31437-26-header Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-31437-26) Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-31437-26) state:2 Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-31437-26-header Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-31437-26-header Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-31437-26-header Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-31437-26) Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [31437] Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-31437-26-header Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-31437-26-header Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-31437-26-header Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:46:55 [31437] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-31437-26) Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-31437-26) state:2 Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-31437-26-header Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-31437-26-header Feb 17 18:46:55 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-31437-26-header lrmd: 2016/02/17_18:46:55 DEBUG: p_rabbitmq-server: monitor: action start. lrmd: 2016/02/17_18:46:55 INFO: p_rabbitmq-server: get_monitor(): CHECK LEVEL IS: 30 ocf-ns_IPaddr2: 2016/02/17_18:46:55 INFO: ARPING 10.109.1.2 from 10.109.1.4 br-ex Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.756ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.542ms Unicast reply from 10.109.1.2 [92:8E:7A:63:5B:7C] 0.543ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:46:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:31201 - exited with rc=0 Feb 17 18:46:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:31201:stderr [ -- empty -- ] Feb 17 18:46:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__vrouter_pub_monitor_5000:31201:stdout [ -- empty -- ] Feb 17 18:46:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 pid:31201 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:55 [6668] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(31421) = 0: Resource temporarily unavailable (11) lrmd: 2016/02/17_18:46:55 INFO: p_rabbitmq-server: get_monitor(): get_status() returns 0. lrmd: 2016/02/17_18:46:55 INFO: p_rabbitmq-server: get_monitor(): also checking if we are master. Feb 17 18:46:56 [6668] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_rabbitmq-server_monitor_30000 Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-31811-26) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [31811] Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f761349cf10, cpd=0x7f761349f554 Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f761349cf10 Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-6268-31811-26-header Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-6268-31811-26-header Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-6268-31811-26-header Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-31811-26) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-31811-26) state:2 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-31811-26-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-31811-26-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-31811-26-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (6268-31811-4) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [31811] Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f761349cf10 Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-6268-31811-4-header Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-6268-31811-4-header Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-6268-31811-4-header Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:46:57 [31811] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (6268-31811-4) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-31811-4) state:2 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f761349cf10 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-6268-31811-4-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-6268-31811-4-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-6268-31811-4-header lrmd: 2016/02/17_18:46:57 DEBUG: p_rabbitmq-server: monitor: action start. lrmd: 2016/02/17_18:46:57 INFO: p_rabbitmq-server: get_monitor(): CHECK LEVEL IS: 0 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync notice [MAIN ] sig_exit_handler Node was shut down by a signal Feb 17 18:46:57 [6267] node-3.test.domain.local corosync notice [SERV ] service_exit_schedwrk_handler Unloading all Corosync service engines. Feb 17 18:46:57 [6267] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_withdraw withdrawing server sockets Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_unref qb_ipcs_unref() - destroying Feb 17 18:46:57 [6267] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync vote quorum service v1.0 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_withdraw withdrawing server sockets Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_unref qb_ipcs_unref() - destroying Feb 17 18:46:57 [6267] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync configuration map access Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-18) state:2 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cfg-response-6268-6556-18-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cfg-event-6268-6556-18-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cfg-request-6268-6556-18-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_withdraw withdrawing server sockets Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_unref qb_ipcs_unref() - destroying Feb 17 18:46:57 [6267] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync configuration service Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-21) state:2 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f76137a1570 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-6707-21-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-6707-21-header Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: qb_ipc_us_ready: poll(fd 7) got POLLHUP Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-6707-21-header Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: error: pcmk_cpg_dispatch: Connection to the CPG API failed: Library error (2) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6655-25) state:2 Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: error: crmd_cs_destroy: connection terminated Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761349f630 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-6655-25-header Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_ipc_us_ready: poll(fd 6) got POLLHUP Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-6655-25-header Feb 17 18:46:57 [6655] node-3.test.domain.local cib: error: pcmk_cpg_dispatch: Connection to the CPG API failed: Library error (2) Feb 17 18:46:57 [6655] node-3.test.domain.local cib: error: cib_cs_destroy: Corosync connection lost! Exiting. Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-6655-25-header Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: uninitializeCib: Deallocating the CIB. Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6678-24) state:2 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f7613198910 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-6678-24-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-6678-24-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-6678-24-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6656-22) state:2 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761359e170 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: uninitializeCib: The CIB has been deallocated. Feb 17 18:46:57 [6655] node-3.test.domain.local cib: info: terminate_cib: cib_cs_destroy: Exiting fast... Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-6656-22-header Feb 17 18:46:57 [6655] node-3.test.domain.local cib: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-6678-13) state:2 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-6656-22-header Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-6678-13-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-6656-22-header Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-6678-13-header Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-6678-13-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-19) state:2 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:57 [6556] node-3.test.domain.local pacemakerd: debug: qb_ipc_us_ready: poll(fd 9) got POLLHUP Feb 17 18:46:57 [6556] node-3.test.domain.local pacemakerd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:46:57 [6556] node-3.test.domain.local pacemakerd: error: pcmk_cpg_dispatch: Connection to the CPG API failed: Library error (2) Feb 17 18:46:57 [6556] node-3.test.domain.local pacemakerd: error: mcp_cpg_destroy: Connection destroyed Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f761319bfa0 Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-6656-12) state:2 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-6656-12-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-6268-6556-19-header Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-6656-12-header Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-6656-12-header Feb 17 18:46:57 [6655] node-3.test.domain.local cib: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:46:57 [6556] node-3.test.domain.local pacemakerd: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6655-6707-11) state:2 Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_shm-response-6655-6707-11-header Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_shm-event-6655-6707-11-header Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_shm-request-6655-6707-11-header Feb 17 18:46:57 [6655] node-3.test.domain.local cib: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:46:57 [6655] node-3.test.domain.local cib: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:46:57 [6655] node-3.test.domain.local cib: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: debug: qb_ipc_us_ready: poll(fd 10) got POLLHUP Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: debug: qb_ipc_us_ready: poll(fd 10) got POLLHUP Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: error: crm_ipc_read: Connection to cib_rw failed Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: error: mainloop_gio_callback: Connection to cib_rw[0x1f99100] closed (I/O condition=17) Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: debug: qb_ipc_us_ready: poll(fd 10) got POLLHUP Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-6678-13-data: No such file or directory (2) Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-6678-13-header: No such file or directory (2) Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-6678-13-data: No such file or directory (2) Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-6678-13-header: No such file or directory (2) Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-6678-13-data: No such file or directory (2) Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-6678-13-header: No such file or directory (2) Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: error: attrd_cib_connection_destroy: Connection to the CIB terminated... Feb 17 18:46:57 [6678] node-3.test.domain.local attrd: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: qb_ipc_us_ready: poll(fd 8) got POLLHUP Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: qb_ipc_us_ready: poll(fd 8) got POLLHUP Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: error: crm_ipc_read: Connection to cib_rw failed Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: error: mainloop_gio_callback: Connection to cib_rw[0x1b3ece0] closed (I/O condition=17) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-6268-6556-19-header Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-6268-6556-19-header Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: qb_ipc_us_ready: poll(fd 13) got POLLHUP Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-6707-9-data: Success (0) Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-attrd-request-6678-6707-9-header: Success (0) Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-6707-9-data: Success (0) Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-attrd-response-6678-6707-9-header: Success (0) Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-6707-9-data: Success (0) Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-attrd-event-6678-6707-9-header: Success (0) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_withdraw withdrawing server sockets Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_unref qb_ipcs_unref() - destroying Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: stonith_api_signoff: Signing out of the STONITH Service Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync cluster closed process group service v1.01 Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-stonith-ng-request-6656-6707-10-header Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-stonith-ng-response-6656-6707-10-header Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-stonith-ng-event-6656-6707-10-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6707-23) state:2 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7f76134a4d40 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: debug: get_xpath_object: No match for //st_notify_disconnect in /notify Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-6268-6707-23-header Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: info: tengine_stonith_connection_destroy: Fencing daemon disconnected Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: notice: crmd_exit: Forcing immediate exit: Link has been severed (67) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-6268-6707-23-header Feb 17 18:46:57 [6707] node-3.test.domain.local crmd: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-6268-6707-23-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(6268-6556-20) state:2 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7f761319aac0 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-6268-6556-20-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-6268-6556-20-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-6268-6556-20-header Feb 17 18:46:57 [6267] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_withdraw withdrawing server sockets Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [QB ] qb_ipcs_unref qb_ipcs_unref() - destroying Feb 17 18:46:57 [6267] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync cluster quorum service v0.1 Feb 17 18:46:57 [6267] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync profile loading service Feb 17 18:46:57 [6267] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync watchdog service Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: qb_ipcs_dispatch_connection_request: HUP conn (6668-6707-7) Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6668-6707-7) state:2 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:119 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: qb_ipcs_event_sendv: new_event_notification (6668-6707-7): Bad file descriptor (9) Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_vrouter_monitor_30000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:37 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_heat-engine_monitor_20000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:118 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation ping_vip__public_monitor_20000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:ping_vip__public action:monitor call_id:140 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: qb_ipc_us_ready: poll(fd 8) got POLLHUP Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-6656-12-data: No such file or directory (2) Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_neutron-l3-agent_monitor_20000 Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6655-6656-12-header: No such file or directory (2) Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-6656-12-data: No such file or directory (2) Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6655-6656-12-header: No such file or directory (2) Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:124 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-6656-12-data: No such file or directory (2) Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6655-6656-12-header: No such file or directory (2) Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_mysql_monitor_60000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_mysql action:monitor call_id:143 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: notice: cib_connection_destroy: Connection to the CIB terminated. Shutting down. Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_neutron-metadata-agent_monitor_60000 Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: info: stonith_shutdown: Terminating with 1 clients Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-metadata-agent action:monitor call_id:131 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6656-6707-10) state:2 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_ntp_monitor_20000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:136 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-stonith-ng-response-6656-6707-10-header Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation vip__vrouter_pub_monitor_5000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter_pub action:monitor call_id:41 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-stonith-ng-event-6656-6707-10-header Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-stonith-ng-request-6656-6707-10-header Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_dns_monitor_20000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:117 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_conntrackd_monitor_27000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:123 exit-code:8 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_haproxy_monitor_30000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:45 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_neutron-dhcp-agent_monitor_20000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:125 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_rabbitmq-server_monitor_103000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: services_action_cancel: Cancelling in-flight op: performing early termination of p_rabbitmq-server_monitor_103000 (pid=31421) Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: operation_finished: p_rabbitmq-server_monitor_103000:31421 - terminated with signal 9 Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: info: main: Done Feb 17 18:46:57 [6656] node-3.test.domain.local stonithd: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_103000:31421:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_103000:31421:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_103000:31421:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_103000:31421:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_103000:31421:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_103000:31421:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_monitor_103000:31421:stdout [ -- empty -- ] Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_rabbitmq-server_monitor_103000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_rabbitmq-server action:monitor call_id:138 pid:31421 exit-code:1 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_rabbitmq-server_monitor_30000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: services_action_cancel: Cancelling in-flight op: performing early termination of p_rabbitmq-server_monitor_30000 (pid=31786) Feb 17 18:46:57 [6267] node-3.test.domain.local corosync debug [TOTEM ] memb_leave_message_send sending join/leave message Feb 17 18:46:57 [6267] node-3.test.domain.local corosync notice [MAIN ] _corosync_exit_error Corosync Cluster Engine exiting normally Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: operation_finished: p_rabbitmq-server_monitor_30000:31786 - terminated with signal 9 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:31786:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:31786:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:31786:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:31786:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:31786:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_30000:31786:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_monitor_30000:31786:stdout [ -- empty -- ] Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_rabbitmq-server_monitor_30000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_rabbitmq-server action:monitor call_id:139 pid:31786 exit-code:1 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation vip__vrouter_monitor_5000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__vrouter action:monitor call_id:40 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:127 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/33b25540-7772-40d4-b8de-ed3f21e9e5ee failed Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: crm_client_destroy: Destroying 17 events Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-lrmd-response-6668-6707-7-header Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-lrmd-event-6668-6707-7-header Feb 17 18:46:57 [6668] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-lrmd-request-6668-6707-7-header Feb 17 18:47:00 [6692] node-3.test.domain.local pengine: info: crm_signal_dispatch: Invoking handler for signal 15: Terminated Feb 17 18:47:00 [6692] node-3.test.domain.local pengine: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:47:00 [6692] node-3.test.domain.local pengine: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:47:00 [6692] node-3.test.domain.local pengine: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:47:00 [6668] node-3.test.domain.local pacemaker_remoted: info: crm_signal_dispatch: Invoking handler for signal 15: Terminated Feb 17 18:47:00 [6668] node-3.test.domain.local pacemaker_remoted: info: lrmd_shutdown: Terminating with 0 clients Feb 17 18:47:00 [6668] node-3.test.domain.local pacemaker_remoted: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:47:00 [6668] node-3.test.domain.local pacemaker_remoted: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:47:00 [6668] node-3.test.domain.local pacemaker_remoted: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [MAIN ] main Corosync Cluster Engine ('2.3.4'): started and ready to provide service. Feb 17 18:47:35 [5319] node-3.test.domain.local corosync info [MAIN ] main Corosync built-in features: dbus testagents rdma watchdog augeas pie relro bindnow Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize Token Timeout (3650 ms) retransmit timeout (357 ms) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize token hold (275 ms) retransmits before loss (10 retrans) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize join (60 ms) send_join (0 ms) consensus (3600 ms) merge (200 ms) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize downcheck (1000 ms) fail to recv const (2500 msgs) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize seqno unchanged const (30 rotations) Maximum network MTU 1401 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize window size per rotation (50 messages) maximum messages per rotation (20 messages) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize missed count const (5 messages) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize send threads (2 threads) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token expired timeout (357 ms) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token problem counter (2000 ms) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP threshold (10 problem count) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP multicast threshold (100 problem count) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP automatic recovery check timeout (1000 ms) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP mode set to none. Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize heartbeat_failures_allowed (0) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize max_network_delay (50 ms) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [TOTEM ] totemnet_instance_initialize Initializing transport (UDP/IP Unicast). Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [TOTEM ] init_nss Initializing transmit/receive security (NSS) crypto: none hash: none Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [TOTEM ] timer_function_netif_check_timeout The network interface [192.168.0.4] is now up. Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] main_iface_change_fn Created or loaded sequence id 28.192.168.0.4 for this ring. Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration map access [0] Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cmap [0] Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:47:35 [5319] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cmap Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration service [1] Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cfg [1] Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:47:35 [5319] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cfg Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync cluster closed process group service v1.01 [2] Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cpg [2] Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:47:35 [5319] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cpg Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync profile loading service [4] Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on pload [4] Feb 17 18:47:35 [5319] node-3.test.domain.local corosync warning [WD ] setup_watchdog No Watchdog, try modprobe Feb 17 18:47:35 [5319] node-3.test.domain.local corosync info [WD ] wd_scan_resources no resources configured. Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync watchdog service [7] Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on wd [7] Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [QUORUM] quorum_exec_init_fn Using quorum provider corosync_votequorum Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_readconfig Reading configuration (runtime: 0) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_readconfig ev_tracking=0, ev_tracking_barrier = 0: expected_votes = 0 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=1, expected_votes=3 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: Yes Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync vote quorum service v1.0 [5] Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on votequorum [5] Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:47:35 [5319] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: votequorum Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync cluster quorum service v0.1 [3] Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on quorum [3] Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:47:35 [5319] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: quorum Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [TOTEM ] totemudpu_member_add adding new UDPU member {192.168.0.6} Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [TOTEM ] totemudpu_member_add adding new UDPU member {192.168.0.4} Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [TOTEM ] totemudpu_member_add adding new UDPU member {192.168.0.5} Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_gather_enter entering GATHER state from 15(interface change). Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_commit_token_create Creating commit token because I am the rep. Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] old_ring_state_save Saving state aru 0 high seq received 0 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [MAIN ] corosync_ring_id_store Storing new sequence id for ring 2c Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_commit_enter entering COMMIT state. Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token got commit token Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter entering RECOVERY state. Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter position [0] member 192.168.0.4: Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter previous ring seq 28 rep 192.168.0.4 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter aru 0 high delivered 0 received flag 1 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter Did not need to originate any messages in recovery. Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token got commit token Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token Sending initial ORF token Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru 0 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token retrans flag count 4 token aru 0 install seq 0 aru 0 0 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] old_ring_state_reset Resetting old ring state Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] deliver_messages_from_recovery_to_regular recovery to regular 1-0 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemudpu_member_set_active Marking UDPU member 192.168.0.4 active Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [MAIN ] member_object_joined Member joined: r(0) ip(192.168.0.4) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_operational_enter entering OPERATIONAL state. Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [TOTEM ] memb_state_operational_enter A new membership (192.168.0.4:44) was formed. Members joined: 3 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 3 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[3]: votes: 1, expected: 3 flags: 8 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: Yes Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=1, expected_votes=3 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [SYNC ] sync_barrier_handler Committing synchronization for corosync configuration map access Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_sync_activate Single node sync -> no action Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [CPG ] downlist_log comparing: sender r(0) ip(192.168.0.4) ; members(old:0 left:0) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [CPG ] downlist_log chosen downlist: sender r(0) ip(192.168.0.4) ; members(old:0 left:0) Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [SYNC ] sync_barrier_handler Committing synchronization for corosync cluster closed process group service v1.01 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: Yes Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 3 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[3]: votes: 1, expected: 3 flags: 8 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: Yes Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=1, expected_votes=3 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 3 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[0]: votes: 0, expected: 0 flags: 0 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [SYNC ] sync_barrier_handler Committing synchronization for corosync vote quorum service v1.0 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=1, expected_votes=3 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [QUORUM] log_view_list Members[1]: 3 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [QUORUM] send_library_notification sending quorum notification to (nil), length = 52 Feb 17 18:47:35 [5319] node-3.test.domain.local corosync notice [MAIN ] corosync_sync_completed Completed service synchronization, ready to provide service. Feb 17 18:47:35 [5319] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-5868-18) Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [5868] Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2947770 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-5868-19) Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [5868] Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7f08d2948620 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7f08d2948620 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_trackstart got trackstart request on 0x7f08d2948620 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_trackstart sending initial status to 0x7f08d2948620 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QUORUM] send_library_notification sending quorum notification to 0x7f08d2948620, length = 52 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-5868-20) Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [5868] Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-5868-20) Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-5868-20) state:2 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cfg-response-5690-5868-20-header Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cfg-event-5690-5868-20-header Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cfg-request-5690-5868-20-header Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_gather_enter entering GATHER state from 11(merge during join). Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_commit_token_create Creating commit token because I am the rep. Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] old_ring_state_save Saving state aru 6 high seq received 6 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [MAIN ] corosync_ring_id_store Storing new sequence id for ring 30 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_commit_enter entering COMMIT state. Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token got commit token Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter entering RECOVERY state. Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter TRANS [0] member 192.168.0.4: Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter position [0] member 192.168.0.4: Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter previous ring seq 2c rep 192.168.0.4 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter aru 6 high delivered 6 received flag 1 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter position [1] member 192.168.0.5: Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter previous ring seq 2c rep 192.168.0.5 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter aru 3a high delivered 3a received flag 1 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_recovery_enter Did not need to originate any messages in recovery. Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token got commit token Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_memb_commit_token Sending initial ORF token Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru 0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token install seq 0 aru 0 high seq received 0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] message_handler_orf_token retrans flag count 4 token aru 0 install seq 0 aru 0 0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] old_ring_state_reset Resetting old ring state Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] deliver_messages_from_recovery_to_regular recovery to regular 1-0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] totemudpu_member_set_active Marking UDPU member 192.168.0.5 active Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [MAIN ] member_object_joined Member joined: r(0) ip(192.168.0.5) Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_state_operational_enter entering OPERATIONAL state. Feb 17 18:47:36 [5319] node-3.test.domain.local corosync notice [TOTEM ] memb_state_operational_enter A new membership (192.168.0.4:48) was formed. Members joined: 2 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [SYNC ] sync_barrier_handler Committing synchronization for corosync configuration map access Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_sync_activate My config version is 0 -> no action Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_joinlist got joinlist message from node 0x2 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [CPG ] downlist_log comparing: sender r(0) ip(192.168.0.4) ; members(old:1 left:0) Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [CPG ] downlist_log comparing: sender r(0) ip(192.168.0.5) ; members(old:1 left:0) Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [CPG ] downlist_log chosen downlist: sender r(0) ip(192.168.0.5) ; members(old:1 left:0) Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [SYNC ] sync_barrier_handler Committing synchronization for corosync cluster closed process group service v1.01 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [CPG ] joinlist_inform_clients joinlist_messages[0] group:crmd\x00, ip:r(0) ip(192.168.0.5) , pid:7403 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [CPG ] joinlist_inform_clients joinlist_messages[1] group:attrd\x00, ip:r(0) ip(192.168.0.5) , pid:7369 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [CPG ] joinlist_inform_clients joinlist_messages[2] group:stonithd\x00, ip:r(0) ip(192.168.0.5) , pid:7346 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [CPG ] joinlist_inform_clients joinlist_messages[3] group:cib\x00, ip:r(0) ip(192.168.0.5) , pid:7337 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [CPG ] joinlist_inform_clients joinlist_messages[4] group:pacemakerd\x00, ip:r(0) ip(192.168.0.5) , pid:7234 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 3 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[3]: votes: 1, expected: 3 flags: 0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=1, expected_votes=3 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 3 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[0]: votes: 0, expected: 0 flags: 0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 2 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[2]: votes: 1, expected: 3 flags: 0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] decode_flags flags: quorate: No Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=2, expected_votes=3 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 2 state=1, votes=1, expected=3 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] get_lowest_node_id lowest node id: 2 us: 3 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] get_highest_node_id highest node id: 3 us: 3 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] are_we_quorate quorum regained, resuming activity Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo got nodeinfo message from cluster node 2 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] message_handler_req_exec_votequorum_nodeinfo nodeinfo message[0]: votes: 0, expected: 0 flags: 0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [SYNC ] sync_barrier_handler Committing synchronization for corosync vote quorum service v1.0 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] recalculate_quorum total_votes=2, expected_votes=3 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 2 state=1, votes=1, expected=3 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] calculate_quorum node 3 state=1, votes=1, expected=3 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] get_lowest_node_id lowest node id: 2 us: 3 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [VOTEQ ] get_highest_node_id highest node id: 3 us: 3 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync notice [QUORUM] quorum_api_set_quorum This node is within the primary component and will provide service. Feb 17 18:47:36 [5319] node-3.test.domain.local corosync notice [QUORUM] log_view_list Members[2]: 3 2 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [QUORUM] send_library_notification sending quorum notification to (nil), length = 56 Feb 17 18:47:36 [5319] node-3.test.domain.local corosync notice [MAIN ] corosync_sync_completed Completed service synchronization, ready to provide service. Feb 17 18:47:36 [5319] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 0 Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-20) Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2a4ccb0 Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-21) Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2b4e5a0 Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-21) Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-21) state:2 Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2b4e5a0 Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-21-header Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-21-header Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-21-header Feb 17 18:47:41 [6666] node-3.test.domain.local pacemakerd: notice: mcp_read_config: Configured corosync to accept connections from group 115: OK (1) Feb 17 18:47:41 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:41 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-20-header Feb 17 18:47:41 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-20-header Feb 17 18:47:41 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-20-header Feb 17 18:47:41 [6666] node-3.test.domain.local pacemakerd: notice: main: Starting Pacemaker 1.1.12 (Build: 561c4cf): generated-manpages agent-manpages ncurses libqb-logging libqb-ipc lha-fencing nagios heartbeat corosync-native snmp libesmtp acls Feb 17 18:47:41 [6666] node-3.test.domain.local pacemakerd: info: main: Maximum core file size is: 18446744073709551615 Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-20) Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-20) state:2 Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2a4ccb0 Feb 17 18:47:41 [6666] node-3.test.domain.local pacemakerd: info: qb_ipcs_us_publish: server name: pacemakerd Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-20-header Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-20-header Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-20-header Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-20) Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: cluster_connect_cfg: Our nodeid: 3 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-21) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f08d2b4e440, cpd=0x7f08d2c4eba4 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-22) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c50030 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_procjoin got procjoin message from cluster node 0x3 (r(0) ip(192.168.0.4) ) for pid 6666 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-22-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-22-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-22-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: crm_get_peer: Created entry c1cfa8e7-df94-413f-ba6c-b60bf8ea20e6/0x20422b0 for node (null)/3 (1 total) Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: crm_get_peer: Node 3 has uuid 3 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3] - corosync-cpg is now online Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: cluster_connect_quorum: Configuring Pacemaker to obtain quorum from Corosync Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-22) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-22) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c50030 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-22-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-22-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-22-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-22) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7f08d2c52750 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7f08d2c52750 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7f08d2c52750 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: notice: cluster_connect_quorum: Quorum acquired Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_trackstart got trackstart request on 0x7f08d2c52750 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_trackstart sending initial status to 0x7f08d2c52750 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QUORUM] send_library_notification sending quorum notification to 0x7f08d2c52750, length = 56 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c54070 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-24) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c50030 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-24-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-24-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-24-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 1 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: crm_get_peer: Created entry 4ebe90e2-6e99-4091-b213-891d158da25c/0x2043890 for node (null)/1 (2 total) Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: crm_get_peer: Node 1 has uuid 1 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-24) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-24) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c50030 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-24-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-24-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-24-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-24) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c50030 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-24-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-24-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-24-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-24) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-24) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c50030 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-24-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-24-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-24-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-24) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c50030 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-24-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-24-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-24-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: crm_get_peer: Created entry 8dc0fd06-b885-4a0a-ba83-9a9b9a175f72/0x2043a80 for node (null)/2 (3 total) Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: crm_get_peer: Node 2 has uuid 2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-24) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-24) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c50030 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-24-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-24-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-24-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-23) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c54070 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c54070 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: crm_get_peer: Node 3 is now known as node-3.test.domain.local Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-23) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c54070 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c52160 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: start_child: Using uid=108 and group=115 for process cib Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: start_child: Forked child 6775 for process cib Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-23) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c52160 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.xml (digest: /var/lib/pacemaker/cib/cib.xml.sig) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c545c0 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: validate_with_relaxng: Creating RNG parser context Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000000100 (was 00000000000000000000000004000000) Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: start_child: Forked child 6776 for process stonith-ng Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-23) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c545c0 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/root Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c545c0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6776-24) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6776] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f08d2c55410, cpd=0x7f08d2c55b64 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: activateCibXml: Triggering CIB write for start op Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: startCib: CIB Initialization completed successfully Feb 17 18:47:42 [6775] node-3.test.domain.local cib: notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-25) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f08d2c52020, cpd=0x7f08d2c55eb4 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000100100 (was 00000000000000000000000000000100) Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: start_child: Forked child 6818 for process lrmd Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-23) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c545c0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_procjoin got procjoin message from cluster node 0x3 (r(0) ip(192.168.0.4) ) for pid 6776 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_procjoin got procjoin message from cluster node 0x3 (r(0) ip(192.168.0.4) ) for pid 6775 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6776-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6776] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6818] node-3.test.domain.local pacemaker_remoted: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/root Feb 17 18:47:42 [6818] node-3.test.domain.local pacemaker_remoted: info: qb_ipcs_us_publish: server name: lrmd Feb 17 18:47:42 [6818] node-3.test.domain.local pacemaker_remoted: info: main: Starting Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c54e50 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-26) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c572d0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-27) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c59060 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-26-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-26-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-26-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000100110 (was 00000000000000000000000000100100) Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: start_child: Using uid=108 and group=115 for process attrd Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: start_child: Forked child 6843 for process attrd Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-26) Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-23-header Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-23-header Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-23-header Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: info: crm_get_peer: Created entry 7fa3b87a-c1db-4348-a36b-1a6700d57a06/0x167b300 for node (null)/3 (1 total) Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: info: crm_get_peer: Node 3 has uuid 3 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3] - corosync-cpg is now online Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: info: init_cs_connection_once: Connection to 'corosync': established Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: info: main: Starting up Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-26) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c572d0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-26-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-26-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-26-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-26) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c572d0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6776-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6776-23) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c54e50 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-23-header Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-27-header Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-27-header Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-27-header Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: crm_get_peer: Created entry 6911f686-53b0-4559-8612-2e3d15a08c21/0x1155cd0 for node (null)/3 (1 total) Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: crm_get_peer: Node 3 has uuid 3 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3] - corosync-cpg is now online Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: init_cs_connection_once: Connection to 'corosync': established Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6776-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6776] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c54cd0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-28) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f08d2c5aca0, cpd=0x7f08d2c5b594 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-27) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-27) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c59060 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-27-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-27-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-27-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-26-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-26-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-26-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000101110 (was 00000000000000000000000000100110) Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: start_child: Using uid=108 and group=115 for process pengine Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: start_child: Forked child 6878 for process pengine Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:47:42 [6878] node-3.test.domain.local pengine: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster Feb 17 18:47:42 [6878] node-3.test.domain.local pengine: debug: main: Init server comms Feb 17 18:47:42 [6878] node-3.test.domain.local pengine: info: qb_ipcs_us_publish: server name: pengine Feb 17 18:47:42 [6878] node-3.test.domain.local pengine: info: main: Starting pengine Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-27) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c59060 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-26) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-26) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c572d0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-26-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-26-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-26-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-26) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c572d0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c58200 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-23-header Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-23-header Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-23-header Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: info: crm_get_peer: Node 3 is now known as node-3.test.domain.local Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6776-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6776-23) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c54cd0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-26-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-26-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-26-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000111110 (was 00000000000000000000000000101110) Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: start_child: Using uid=108 and group=115 for process crmd Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: start_child: Forked child 6905 for process crmd Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6776-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6776] Feb 17 18:47:42 [6905] node-3.test.domain.local crmd: info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster Feb 17 18:47:42 [6905] node-3.test.domain.local crmd: notice: main: CRM Git Version: 561c4cf Feb 17 18:47:42 [6905] node-3.test.domain.local crmd: debug: crmd_init: Starting crmd Feb 17 18:47:42 [6905] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_STARTUP: [ state=S_STARTING cause=C_STARTUP origin=crmd_init ] Feb 17 18:47:42 [6905] node-3.test.domain.local crmd: info: do_log: FSA: Input I_STARTUP from crmd_init() received in state S_STARTING Feb 17 18:47:42 [6905] node-3.test.domain.local crmd: debug: do_startup: Registering Signal Handlers Feb 17 18:47:42 [6905] node-3.test.domain.local crmd: debug: do_startup: Creating CIB and LRM objects Feb 17 18:47:42 [6905] node-3.test.domain.local crmd: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:47:42 [6905] node-3.test.domain.local crmd: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:47:42 [6905] node-3.test.domain.local crmd: info: crm_ipc_connect: Could not establish cib_shm connection: Connection refused (111) Feb 17 18:47:42 [6905] node-3.test.domain.local crmd: debug: cib_native_signon_raw: Connection unsuccessful (0 (nil)) Feb 17 18:47:42 [6905] node-3.test.domain.local crmd: debug: cib_native_signon_raw: Connection to CIB failed: Transport endpoint is not connected Feb 17 18:47:42 [6905] node-3.test.domain.local crmd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c54cd0 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-27-header Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-27-header Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-27-header Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: crm_get_peer: Node 3 is now known as node-3.test.domain.local Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-26) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-26) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c572d0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-26-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-26-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-26-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CPG ] message_handler_req_exec_cpg_procjoin got procjoin message from cluster node 0x3 (r(0) ip(192.168.0.4) ) for pid 6843 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-26) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c572d0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-27) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-27) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c59060 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-27-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-27-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-27-header Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: notice: get_node_name: Could not obtain a node name for corosync nodeid 3 Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: info: crm_get_peer: Created entry a5060fd5-5ea1-4449-8620-892e627f5012/0x8a38d0 for node (null)/3 (1 total) Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: info: crm_get_peer: Node 3 has uuid 3 Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3] - corosync-cpg is now online Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: info: init_cs_connection_once: Connection to 'corosync': established Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-27) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c59060 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c58200 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-23-header Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-23-header Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-23-header Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: info: crm_ipc_connect: Could not establish cib_rw connection: Connection refused (111) Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: cib_native_signon_raw: Connection unsuccessful (0 (nil)) Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: cib_native_signon_raw: Connection to CIB failed: Transport endpoint is not connected Feb 17 18:47:42 [6776] node-3.test.domain.local stonithd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-26-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-26-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-26-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000111310 (was 00000000000000000000000000111110) Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: main: Starting mainloop Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: info: pcmk_quorum_notification: Membership 48: quorum retained (2) Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: pcmk_quorum_notification: Member[0] 3 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c58200 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6776-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6776-23) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c54cd0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-26) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-26) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c572d0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-26-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-26-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-26-header Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-27-header Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-27-header Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-27-header Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: qb_ipcs_us_publish: server name: cib_ro Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: qb_ipcs_us_publish: server name: cib_rw Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: qb_ipcs_us_publish: server name: cib_shm Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: cib_init: Starting cib mainloop Feb 17 18:47:42 [6775] node-3.test.domain.local cib: info: pcmk_cpg_membership: Joined[0.0] cib.3 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c578d0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-27) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-27) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c59060 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-27-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-27-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-27-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c59060 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: notice: crm_update_peer_state: pcmk_quorum_notification: Node node-3.test.domain.local[3] - state is now member (was (null)) Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: pcmk_quorum_notification: Member[1] 2 Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:47:42 [6843] node-3.test.domain.local attrd: info: crm_get_peer: Node 3 is now known as node-3.test.domain.local Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-23) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c578d0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-23) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c578d0 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c58200 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-27) Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c58200 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: crm_get_peer: Created entry 6a687d9c-c063-494e-bcff-3f4bc222700f/0x1158fa0 for node (null)/2 (2 total) Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: crm_get_peer: Node 2 has uuid 2 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: pcmk_cpg_membership: Member[0.0] cib.2 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[2] - corosync-cpg is now online Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c59060 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-27-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-27-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-27-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: info: main: Cluster connection active Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: info: qb_ipcs_us_publish: server name: attrd Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: info: main: Accepting attribute updates Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: notice: main: Starting mainloop... Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: info: pcmk_cpg_membership: Joined[0.0] attrd.3 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5c9b0 Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: info: pcmk_quorum_notification: Obtaining name for new node 2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-27) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-27) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c58200 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-27-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-27-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-27-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-27) Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c58200 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-23) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-23) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c578d0 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: pcmk_cpg_membership: Member[0.1] cib.3 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-23) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c578d0 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5c9b0 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5c980 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-27-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-27-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-27-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: info: crm_get_peer: Created entry 5a9fdab3-1211-4d1d-8733-a7fbded786d5/0x8a5170 for node (null)/2 (2 total) Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: info: crm_get_peer: Node 2 has uuid 2 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: info: pcmk_cpg_membership: Member[0.0] attrd.2 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[2] - corosync-cpg is now online Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-27) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-27) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c58200 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-27-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-27-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-27-header Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: crm_get_peer: Node 2 is now known as node-2.test.domain.local Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-27) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a180 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-23) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-23) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c578d0 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5c980 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-23) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c59060 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-27-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-27-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-27-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: info: pcmk_cpg_membership: Member[0.1] attrd.3 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c57800 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-27) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-27) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a180 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-27-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-27-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-27-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-27) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a180 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:43 [6775] node-3.test.domain.local cib: warning: cib_process_request: Completed cib_modify operation for section status: No such device or address (rc=-6, origin=node-2.test.domain.local/attrd/1049, version=0.155.0) Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: notice: crm_update_peer_state: pcmk_quorum_notification: Node (null)[2] - state is now member (was (null)) Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: info: pcmk_quorum_notification: State of node (null)[1] is still unknown Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: info: crm_get_peer: Node 2 is now known as node-2.test.domain.local Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c57800 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c57800 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-23) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-23) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c59060 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-27-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-27-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-27-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: info: crm_get_peer: Node 2 is now known as node-2.test.domain.local Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6666-23) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6666] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c59060 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-27) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-27) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a180 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-27-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-27-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-27-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:43 [6775] node-3.test.domain.local cib: warning: cib_process_request: Completed cib_modify operation for section status: No such device or address (rc=-6, origin=node-2.test.domain.local/attrd/1051, version=0.155.0) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-27) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a180 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c57800 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-2.test.domain.local now has process list: 00000000000000000000000000111310 (was 00000000000000000000000000000000) Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6775) = 0: Resource temporarily unavailable (11) Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6776) = 0: Resource temporarily unavailable (11) Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6818) = 0: Resource temporarily unavailable (11) Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6843) = 0: Resource temporarily unavailable (11) Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6878) = 0: Resource temporarily unavailable (11) Feb 17 18:47:43 [6666] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6905) = 0: Resource temporarily unavailable (11) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5c980 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6666-23) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-23) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c59060 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6666-23-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6666-23-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6666-23-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-27-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-27-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-27-header Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for var_log_free Feb 17 18:47:43 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation var_log_free=: cib not connected Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-27) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-27) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a180 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-27-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-27-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-27-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:43 [6775] node-3.test.domain.local cib: warning: cib_process_request: Completed cib_modify operation for section status: No such device or address (rc=-6, origin=node-2.test.domain.local/attrd/1053, version=0.155.0) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5c980 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-23) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a180 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-23-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-23-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-23-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:43 [6775] node-3.test.domain.local cib: warning: cib_process_request: Completed cib_modify operation for section status: No such device or address (rc=-6, origin=node-2.test.domain.local/attrd/1055, version=0.155.0) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-23) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-23) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a180 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-23-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-23-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-23-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-23) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c583e0 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-23-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-23-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-23-header Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.0 2 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.1 (null) Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=1 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status: Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:43 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/455, version=0.155.5) Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x116d490 for uid=0 gid=0 pid=6776 id=14ccbbcd-3c2a-4ab0-91e6-b64e07f66bc0 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-6776-12) Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [6776] Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: cib_common_callback_worker: Setting cib_diff_notify callbacks for crmd (14ccbbcd-3c2a-4ab0-91e6-b64e07f66bc0): on Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: notice: setup_cib: Watching for stonith topology changes Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: qb_ipcs_us_publish: server name: stonith-ng Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: main: Starting stonithd mainloop Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: pcmk_cpg_membership: Joined[0.0] stonithd.3 Feb 17 18:47:43 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-27-header Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-27-header Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-27-header Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: cluster_connect_quorum: Configuring Pacemaker to obtain quorum from Corosync Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6776-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6776] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-27) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-27) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c58cb0 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-27-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-27-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-27-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-27) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7f08d2c58cb0 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7f08d2c58cb0 Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: notice: cluster_connect_quorum: Quorum acquired Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7f08d2c58cb0 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_trackstart got trackstart request on 0x7f08d2c58cb0 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_trackstart sending initial status to 0x7f08d2c58cb0 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QUORUM] send_library_notification sending quorum notification to 0x7f08d2c58cb0, length = 56 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-26-header Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-26-header Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-26-header Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: crm_get_peer: Created entry 52ad4d0b-c221-4942-9ebf-ca7afce2aafd/0x167d3f0 for node (null)/2 (2 total) Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: crm_get_peer: Node 2 has uuid 2 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: pcmk_cpg_membership: Member[0.0] stonithd.2 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[2] - corosync-cpg is now online Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: st_peer_update_callback: Broadcasting our uname because of node 2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-29) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5da50 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6776-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6776-26) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6776-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6776] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-30) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5f250 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-26-header Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-26-header Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-26-header Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6776-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6776-26) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6776-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6776] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ccd0 Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-30-header Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-30-header Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-30-header Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 1 Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: info: crm_get_peer: Created entry e760513e-1276-4d4b-ae8e-1a499f834db5/0x1414580 for node (null)/1 (2 total) Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: info: crm_get_peer: Node 1 has uuid 1 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-30) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-30) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5f250 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-30-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-30-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-30-header Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-26-header Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-26-header Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-26-header Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: pcmk_cpg_membership: Member[0.1] stonithd.3 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: init_cib_cache_cb: Updating device list from the cib: init Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: cib_devices_update: Updating devices to version 0.155.5 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: unpack_config: Stop all active resources: false Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: unpack_config: Default stickiness: 0 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: apply_system_health: Applying automated node health strategy: migrate-on-red Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: apply_system_health: Node node-1.test.domain.local has an combined system health of 0 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: apply_system_health: Node node-2.test.domain.local has an combined system health of 0 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: apply_system_health: Node node-3.test.domain.local has an combined system health of 0 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: info: crm_get_peer: Node 2 is now known as node-2.test.domain.local Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: st_peer_update_callback: Broadcasting our uname because of node 2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6776-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6776-26) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ccd0 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6776-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6776] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ccd0 Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-26-header Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-26-header Feb 17 18:47:43 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-26-header Feb 17 18:47:43 [6905] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6776-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6776-26) state:2 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ccd0 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-26-header Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6776-26) Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6776] Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ccd0 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-30) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5f300 Feb 17 18:47:44 [6776] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:44 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-26-header Feb 17 18:47:44 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-26-header Feb 17 18:47:44 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6776-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6776-26) state:2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ccd0 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-30-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-30-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-30-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6776-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6776] Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-30) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-30) state:2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5f300 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-30-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-30-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-30-header Feb 17 18:47:44 [6776] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:44 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-26-header Feb 17 18:47:44 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-26-header Feb 17 18:47:44 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6776-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6776-26) state:2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6776-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6776-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6776-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: crm_get_peer: Created entry 4fb1941e-0dcb-4c12-866f-1ac11a84ef45/0x1414820 for node (null)/2 (3 total) Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: crm_get_peer: Node 2 has uuid 2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-29-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-29-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-29-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-29) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-29) state:2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5da50 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-29-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-29-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-29-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-29) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5f130 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-29-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-29-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-29-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 1 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 1 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-29) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-29) state:2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5f130 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-29-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-29-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-29-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-29) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c608c0 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-29-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-29-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-29-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-29) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-29) state:2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c608c0 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-29-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-29-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-29-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-29) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5f130 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-29-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-29-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-29-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-29) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-29) state:2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5f130 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-29-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-29-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-29-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: do_ha_control: Connected to the cluster Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: do_lrm_control: Connecting to the LRM Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: lrmd_ipc_connect: Connecting to lrmd Feb 17 18:47:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section nodes to master (origin=local/crmd/3) Feb 17 18:47:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: crm_client_new: Connecting 0x13f2b10 for uid=108 gid=115 pid=6905 id=0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa Feb 17 18:47:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: handle_new_connection: IPC credentials authenticated (6818-6905-7) Feb 17 18:47:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: qb_ipcs_shm_connect: connecting to client [6905] Feb 17 18:47:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: do_lrm_control: LRM connection established Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: do_started: Delaying start, no membership data (0000000000100000) Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: register_fsa_input_adv: Stalling the FSA pending further input: source=do_started cause=C_FSA_INTERNAL data=(nil) queue=0 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: s_crmd_fsa: Exiting the FSA: queue=0, fsa_actions=0x2, stalled=true Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: pcmk_quorum_notification: Membership 48: quorum retained (2) Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: pcmk_quorum_notification: Member[0] 3 Feb 17 18:47:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60040 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: notice: crm_update_peer_state: pcmk_quorum_notification: Node node-3.test.domain.local[3] - state is now member (was (null)) Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: peer_update_callback: node-3.test.domain.local is now member (was (null)) Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: pcmk_quorum_notification: Member[1] 2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-3.test.domain.local/crmd/3, version=0.155.5) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60040 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: pcmk_quorum_notification: Obtaining name for new node 2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: notice: crm_update_peer_state: pcmk_quorum_notification: Node (null)[2] - state is now member (was (null)) Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: pcmk_quorum_notification: State of node (null)[1] is still unknown Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: post_cache_update: Updated cache after membership event 48. Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: post_cache_update: post_cache_update added action A_ELECTION_CHECK to the FSA Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: do_started: Delaying start, Config not read (0000000000000040) Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: register_fsa_input_adv: Stalling the FSA pending further input: source=do_started cause=C_FSA_INTERNAL data=(nil) queue=0 Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: s_crmd_fsa: Exiting the FSA: queue=0, fsa_actions=0x200000002, stalled=true Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: config_query_callback: Call 4 : Parsing CIB options Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: config_query_callback: Shutdown escalation occurs after: 1200000ms Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: config_query_callback: Checking for expired actions every 190000ms Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: do_started: Init server comms Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: qb_ipcs_us_publish: server name: crmd Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: notice: do_started: The local CRM is operational Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: do_election_check: Ignore election check: we not in an election Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_STARTING cause=C_FSA_INTERNAL origin=do_started ] Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: info: do_log: FSA: Input I_PENDING from do_started() received in state S_STARTING Feb 17 18:47:44 [6905] node-3.test.domain.local crmd: notice: do_state_transition: State transition S_STARTING -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_started ] Feb 17 18:47:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: do_cl_join_query: Querying for a DC Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=19 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: info: pcmk_cpg_membership: Joined[0.0] crmd.3 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: info: corosync_node_name: Unable to get node name for nodeid 2 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: notice: get_node_name: Could not obtain a node name for corosync nodeid 2 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: info: pcmk_cpg_membership: Member[0.0] crmd.2 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[2] - corosync-cpg is now online Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: info: pcmk_cpg_membership: Member[0.1] crmd.3 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: info: crm_get_peer: Node 2 is now known as node-2.test.domain.local Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: info: peer_update_callback: node-2.test.domain.local is now member Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: handle_request: Raising I_JOIN_OFFER: join-2 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ] Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: info: update_dc: Set DC to node-2.test.domain.local (3.0.9) Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: join_query_callback: Respond to join offer join-2 from node-2.test.domain.local Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: handle_request: Raising I_JOIN_OFFER: join-2 Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ] Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA Feb 17 18:47:45 [6905] node-3.test.domain.local crmd: debug: te_connect_stonith: Attempting connection to fencing daemon... Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_process_replace: Digest matched on replace from node-2.test.domain.local: eefc6ca1920fa7794a82ac9badca49e1 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_process_replace: Replaced 0.155.5 with 0.155.729 from node-2.test.domain.local Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: __xml_diff_object: Moved node_state@crm-debug-origin (2 -> 4) Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: __xml_diff_object: lrm_resource.vip__management moved from 1 to 0 - 20916576 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: __xml_diff_object: lrm_resource.vip__vrouter_pub moved from 2 to 1 - 21012816 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: __xml_diff_object: lrm_resource.sysinfo_node-2.test.domain.local moved from 0 to 2 - 21099440 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: __xml_diff_object: lrm_resource.vip__vrouter moved from 1 to 2 - 21113408 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: __xml_diff_object: Moved node_state@uname (1 -> 4) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.5 2 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.729 (null) Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=729, @cib-last-written=Wed Feb 17 14:21:54 2016, @dc-uuid=2 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']: @in_ccm=true, @crmd=online, @crm-debug-origin=do_update_resource, @join=member, @expected=member Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: +~ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='vip__vrouter_pub'] moved to offset 10 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='sysinfo_node-2.test.domain.local']: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']: @in_ccm=true, @uname=node-3.test.domain.local, @join=member, @expected=member Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: activateCibXml: Triggering CIB write for cib_replace op Feb 17 18:47:45 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.729 to 0.155.5 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_replace_notify: Replaced: 0.155.5 -> 0.155.729 from node-2.test.domain.local Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_replace operation for section 'all': OK (rc=0, origin=node-2.test.domain.local/crmd/457, version=0.155.729) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-2.test.domain.local/crmd/458, version=0.155.729) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-2.test.domain.local/crmd/459, version=0.155.729) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_delete op for //node_state[@uname='node-2.test.domain.local']/lrm (/cib/status/node_state[1]/lrm) Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.729 2 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.730 (null) Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: -- /cib/status/node_state[@id='2']/lrm[@id='2'] Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=730 Feb 17 18:47:45 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.730 to 0.155.729 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='node-2.test.domain.local']/lrm: OK (rc=0, origin=node-2.test.domain.local/crmd/460, version=0.155.730) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-93.raw Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: write_cib_contents: Writing CIB to disk Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: write_cib_contents: Wrote version 0.155.0 of the CIB to disk (digest: 1c289e4b1d826b05a4d332eba0c00353) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: write_cib_contents: Wrote digest 1c289e4b1d826b05a4d332eba0c00353 to disk Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.XaIjTB (digest: /var/lib/pacemaker/cib/cib.yGl9xA) Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.730 2 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.731 (null) Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=731 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_lrm_query_internal Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']: Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:45 [6775] node-3.test.domain.local cib: debug: write_cib_contents: Activating /var/lib/pacemaker/cib/cib.XaIjTB Feb 17 18:47:45 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.731 to 0.155.730 Feb 17 18:47:45 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/461, version=0.155.731) Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: crm_client_new: Connecting 0x1909680 for uid=108 gid=115 pid=6905 id=4ba93b2b-0ce6-4d2e-a9de-aa655197d719 Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: handle_new_connection: IPC credentials authenticated (6776-6905-10) Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: qb_ipcs_shm_connect: connecting to client [6905] Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: stonith_command: Processing register 11 from crmd.6905 ( 0) Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: stonith_command: Processed register from crmd.6905: OK (0) Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: stonith_api_signon: Connection to STONITH successful Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: stonith_command: Processing st_notify 12 from crmd.6905 ( 0) Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: handle_request: Setting st_notify_disconnect callbacks for crmd.6905 (4ba93b2b-0ce6-4d2e-a9de-aa655197d719): ON Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: stonith_command: Processed st_notify from crmd.6905: OK (0) Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: stonith_command: Processing st_notify 13 from crmd.6905 ( 0) Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: handle_request: Setting st_notify_fence callbacks for crmd.6905 (4ba93b2b-0ce6-4d2e-a9de-aa655197d719): ON Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: stonith_command: Processed st_notify from crmd.6905: OK (0) Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: join_query_callback: Respond to join offer join-2 from node-2.test.domain.local Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: do_cib_replaced: Updating the CIB after a replace: DC=false Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: handle_request: Raising I_JOIN_RESULT: join-2 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ] Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: do_cl_join_finalize_respond: Confirming join join-2: join_ack_nack Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: do_cl_join_finalize_respond: join-2: Join complete. Sending local LRM status to node-2.test.domain.local Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: info: erase_status_tag: Deleting xpath: //node_state[@uname='node-3.test.domain.local']/transient_attributes Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: info: update_attrd_helper: Connecting to attrd... 5 retries remaining Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_delete operation for section //node_state[@uname='node-3.test.domain.local']/transient_attributes to master (origin=local/crmd/8) Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x8a83d0 for uid=108 gid=115 pid=6905 id=4ae2d009-632d-46fb-a255-8f045bc278b6 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-6905-9) Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [6905] Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: attrd_update_delegate: Sent update: terminate=(null) for node-3.test.domain.local Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: attrd_update_delegate: Sent update: shutdown=0 for node-3.test.domain.local Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crmd: terminate= Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for terminate Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null) Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crmd: shutdown=0 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for shutdown Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: (null), Stored: (null) Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of shutdown is 0 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: shutdown (0) Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation shutdown=0: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60040 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ] Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: info: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-30) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ed20 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_delete op for //node_state[@uname='node-3.test.domain.local']/transient_attributes (/cib/status/node_state[2]/transient_attributes) Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.731 2 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.732 (null) Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: -- /cib/status/node_state[@id='3']/transient_attributes[@id='3'] Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=732 Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.732 to 0.155.731 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='node-3.test.domain.local']/transient_attributes: OK (rc=0, origin=node-3.test.domain.local/crmd/8, version=0.155.732) Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: erase_xpath_callback: Deletion of "//node_state[@uname='node-3.test.domain.local']/transient_attributes": OK (rc=0) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60040 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-30-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-30-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-30-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: notice: attrd_local_callback: Sending full refresh (origin=crmd) Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: shutdown (0) Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation shutdown=0: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-30) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-30) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ed20 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-30-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-30-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-30-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60040 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: handle_request: Raising I_JOIN_OFFER: join-2 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_NOT_DC cause=C_HA_MESSAGE origin=route_message ] Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: notice: do_state_transition: State transition S_NOT_DC -> S_PENDING [ input=I_JOIN_OFFER cause=C_HA_MESSAGE origin=route_message ] Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: info: update_dc: Unset DC. Was node-2.test.domain.local Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: info: update_dc: Set DC to node-2.test.domain.local (3.0.9) Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-30) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ece0 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1057, version=0.155.732) Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: join_query_callback: Respond to join offer join-2 from node-2.test.domain.local Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60040 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-30-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-30-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-30-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-30) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-30) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ece0 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-30-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-30-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-30-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1059, version=0.155.732) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_replace: Digest matched on replace from node-2.test.domain.local: 025839f54f7310ac5abb282071f4fbd7 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_replace: Replaced 0.155.732 with 0.155.732 from node-2.test.domain.local Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: activateCibXml: Triggering CIB write for cib_replace op Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_replace operation for section 'all': OK (rc=0, origin=node-2.test.domain.local/crmd/463, version=0.155.732) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60040 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-2.test.domain.local/crmd/464, version=0.155.732) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-29-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-29-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60040 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60040 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-2.test.domain.local/crmd/465, version=0.155.732) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-29-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-29-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-29-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: handle_request: Raising I_JOIN_RESULT: join-2 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ] Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: do_cl_join_finalize_respond: Confirming join join-2: join_ack_nack Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60040 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-94.raw Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: write_cib_contents: Writing CIB to disk Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: do_cl_join_finalize_respond: join-2: Join complete. Sending local LRM status to node-2.test.domain.local Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: write_cib_contents: Wrote version 0.155.0 of the CIB to disk (digest: 3323a5b298a40cbb8312e9dd537fb30f) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ] Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: info: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING Feb 17 18:47:46 [6905] node-3.test.domain.local crmd: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ] Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: notice: attrd_local_callback: Sending full refresh (origin=crmd) Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: shutdown (0) Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation shutdown=0: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: write_cib_contents: Wrote digest 3323a5b298a40cbb8312e9dd537fb30f to disk Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.jMQL7B (digest: /var/lib/pacemaker/cib/cib.oqkzLE) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: write_cib_contents: Activating /var/lib/pacemaker/cib/cib.jMQL7B Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_delete op for //node_state[@uname='node-2.test.domain.local']/lrm (/cib/status/node_state[1]/lrm) Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.732 2 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.733 (null) Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: -- /cib/status/node_state[@id='2']/lrm[@id='2'] Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=733 Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.733 to 0.155.732 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='node-2.test.domain.local']/lrm: OK (rc=0, origin=node-2.test.domain.local/crmd/466, version=0.155.733) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.733 2 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.734 (null) Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=734 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']: Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.734 to 0.155.733 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/467, version=0.155.734) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_delete op for //node_state[@uname='node-3.test.domain.local']/lrm (/cib/status/node_state[2]/lrm) Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.734 2 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.735 (null) Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: -- /cib/status/node_state[@id='3']/lrm[@id='3'] Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=735 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='node-3.test.domain.local']/lrm: OK (rc=0, origin=node-2.test.domain.local/crmd/468, version=0.155.735) Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.735 to 0.155.734 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60020 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.735 2 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.736 (null) Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=736 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']: @crm-debug-origin=do_lrm_query_internal Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']: Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/469, version=0.155.736) Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.736 to 0.155.735 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1061, version=0.155.736) Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for rabbit_get_alarms_timeouts Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation rabbit_get_alarms_timeouts=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60020 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=node-2.test.domain.local/crmd/472, version=0.155.736) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.736 2 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.737 (null) Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=737 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']: @crm-debug-origin=do_state_transition Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/473, version=0.155.737) Feb 17 18:47:46 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.737 to 0.155.736 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section cib: OK (rc=0, origin=node-2.test.domain.local/crmd/474, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1063, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for fail-count-p_mysql Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation fail-count-p_mysql=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1065, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1067, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for free_swap Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation free_swap=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1069, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5efa0 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for last-failure-p_neutron-l3-agent Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation last-failure-p_neutron-l3-agent=: cib not connected Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1071, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5efa0 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ff10 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1073, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ff10 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ff10 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for gtid Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation gtid=: cib not connected Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1075, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ff10 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1077, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for rabbit_list_channels_timeouts Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation rabbit_list_channels_timeouts=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1079, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1081, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for fail-count-p_neutron-l3-agent Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation fail-count-p_neutron-l3-agent=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1083, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1085, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for cpu_cores Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation cpu_cores=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1087, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1089, version=0.155.737) Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for arch Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation arch=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1091, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for ram_total Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation ram_total=: cib not connected Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1093, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for ram_free Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation ram_free=: cib not connected Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1095, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for rabbit_list_queues_timeouts Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation rabbit_list_queues_timeouts=: cib not connected Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1097, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1099, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for cpu_load Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation cpu_load=: cib not connected Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1101, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5e7b0 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5e7b0 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60040 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1103, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60040 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for root_free Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation root_free=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ddc0 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1105, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ddc0 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for var_lib_mysql_free Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation var_lib_mysql_free=: cib not connected Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1107, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1109, version=0.155.737) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for last-failure-p_mysql Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation last-failure-p_mysql=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for cpu_info Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation cpu_info=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for os Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation os=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation shutdown=0: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for master-p_rabbitmq-server Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation master-p_rabbitmq-server=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for master-p_conntrackd Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation master-p_conntrackd=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for cpu_speed Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation cpu_speed=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for probe_complete Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation probe_complete=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation var_log_free=: cib not connected Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'sysinfo_node-1.test.domain.local' not found (0 active resources) Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'sysinfo_node-1.test.domain.local' to the rsc list (1 active resources) Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=5:125:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=sysinfo_node-1.test.domain.local_monitor_0 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=5, reply=1, notify=0, exit=4202464 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:sysinfo_node-1.test.domain.local action:monitor call_id:5 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(8476) = 0: Resource temporarily unavailable (11) Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-1.test.domain.local_monitor_0:8476 - exited with rc=7 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: sysinfo_node-1.test.domain.local_monitor_0:8476:stderr [ rm: cannot remove '/var/run//SysInfo-sysinfo_node-1.test.domain.local': No such file or directory ] Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-1.test.domain.local_monitor_0:8476:stdout [ -- empty -- ] Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-1.test.domain.local action:monitor call_id:5 pid:8476 exit-code:7 exec-time:19ms queue-time:0ms Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_vrouter' not found (1 active resources) Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_vrouter:1' not found (1 active resources) Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_vrouter' to the rsc list (2 active resources) Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=6:125:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_vrouter_monitor_0 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=10, reply=1, notify=0, exit=4202464 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource sysinfo_node-1.test.domain.local after monitor op complete (interval=0) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_vrouter action:monitor call_id:10 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed SysInfo_meta-data_0 process 8483 exited with rc=0 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/11) Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation sysinfo_node-1.test.domain.local_monitor_0: not running (node=node-3.test.domain.local, call=5, rc=7, cib-update=11, confirmed=true) Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: node-3.test.domain.local-sysinfo_node-1.test.domain.local_monitor_0:5 [ rm: cannot remove '/var/run//SysInfo-sysinfo_node-1.test.domain.local': No such file or directory\n ] Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'sysinfo_node-1.test.domain.local' with monitor op Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.737 2 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.738 (null) Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=738 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']: @crm-debug-origin=do_update_resource Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/11, version=0.155.738) Feb 17 18:47:47 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.738 to 0.155.737 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'vip__management' not found (2 active resources) Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'vip__management' to the rsc list (3 active resources) Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=7:125:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=vip__management_monitor_0 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=14, reply=1, notify=0, exit=4202464 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:vip__management action:monitor call_id:14 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'vip__vrouter_pub' not found (3 active resources) Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'vip__vrouter_pub' to the rsc list (4 active resources) Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=8:125:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=vip__vrouter_pub_monitor_0 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=18, reply=1, notify=0, exit=4202464 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:vip__vrouter_pub action:monitor call_id:18 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-ns_vrouter: 2016/02/17_18:47:47 DEBUG: p_vrouter: check_ns(): recieved netns list: Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5fe90 Feb 17 18:47:47 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:47 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:47 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:47 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:47 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:47 [6843] node-3.test.domain.local attrd: info: attrd_perform_update: Delaying operation cpu_load=: cib not connected Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-30) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5f080 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5fe90 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.738 2 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.739 (null) Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=739 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.48 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1111, version=0.155.739) Feb 17 18:47:47 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.739 to 0.155.738 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-30-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-30-header Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-30-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-30) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-30) state:2 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5f080 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-30-header Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'vip__vrouter' not found (4 active resources) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-30-header Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'vip__vrouter' to the rsc list (5 active resources) Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:47 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=9:125:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=vip__vrouter_monitor_0 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=22, reply=1, notify=0, exit=4202464 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-30-header Feb 17 18:47:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:vip__vrouter action:monitor call_id:22 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.739 2 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.740 (null) Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=740 Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='sysinfo_node-2.test.domain.local']: Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:47 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/21, version=0.155.744) Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_speed'] does not exist Feb 17 18:47:47 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.744 to 0.155.743 Feb 17 18:47:47 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for cpu_speed=(null) passed Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_conntrackd'] does not exist Feb 17 18:47:47 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for master-p_conntrackd=(null) passed Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] does not exist Feb 17 18:47:47 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for var_log_free=(null) passed Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:47 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='probe_complete'] does not exist Feb 17 18:47:47 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for probe_complete=(null) passed Feb 17 18:47:47 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 21 for shutdown=0 passed Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-ns_vrouter: 2016/02/17_18:47:47 DEBUG: Setting namespace interface: vr-host-ns IP to: 240.0.0.6/30 ocf-ns_vrouter: 2016/02/17_18:47:48 DEBUG: Creating default route inside the namespace to 240.0.0.5 with metric 10000 ocf-ns_vrouter: 2016/02/17_18:47:48 INFO: net.ipv4.conf.vr-host-base.rp_filter = 2 ocf-ns_vrouter: 2016/02/17_18:47:48 INFO: net.ipv4.conf.all.rp_filter = 2 ocf-ns_vrouter: 2016/02/17_18:47:48 DEBUG: Adding route on the host system to 10.109.0.0/24: 240.0.0.6 ocf-ns_vrouter: 2016/02/17_18:47:48 DEBUG: Adding route on the host system to 192.168.2.0/24: 240.0.0.6 ocf-ns_vrouter: 2016/02/17_18:47:48 DEBUG: Adding route on the host system to 192.168.1.0/24: 240.0.0.6 ocf-ns_vrouter: 2016/02/17_18:47:48 DEBUG: Adding route on the host system to 192.168.0.0/24: 240.0.0.6 ocf-ns_vrouter: 2016/02/17_18:47:48 DEBUG: Adding route on the host system to 10.109.1.0/24: 240.0.0.6 Feb 17 18:47:48 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_0:8485 - exited with rc=0 Feb 17 18:47:48 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_0:8485:stderr [ -- empty -- ] Feb 17 18:47:48 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_0:8485:stdout [ -- empty -- ] Feb 17 18:47:48 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:10 pid:8485 exit-code:0 exec-time:934ms queue-time:1ms Feb 17 18:47:48 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_vrouter after monitor op complete (interval=0) Feb 17 18:47:48 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_vrouter_meta-data_0 process 9393 exited with rc=0 Feb 17 18:47:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/15) Feb 17 18:47:48 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_vrouter_monitor_0: ok (node=node-3.test.domain.local, call=10, rc=0, cib-update=15, confirmed=true) Feb 17 18:47:48 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_vrouter' with monitor op Feb 17 18:47:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:48 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.744 2 Feb 17 18:47:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.745 (null) Feb 17 18:47:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=745 Feb 17 18:47:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:47:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/15, version=0.155.745) Feb 17 18:47:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:48 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.745 to 0.155.744 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.745 2 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.746 (null) Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=746 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='vip__management']: word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.746 2 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.747 (null) Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=747 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='vip__vrouter_pub']: word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.747 2 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.748 (null) Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=748 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='vip__vrouter']: word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=26:126:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_vrouter_monitor_30000 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=23, reply=1, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_vrouter action:monitor call_id:23 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'vip__public' not found (5 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'vip__public' to the rsc list (6 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=9:126:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=vip__public_monitor_0 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=27, reply=1, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:vip__public action:monitor call_id:27 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_conntrackd' not found (6 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_conntrackd:1' not found (6 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_conntrackd' to the rsc list (7 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=10:126:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_monitor_0 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=32, reply=1, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_conntrackd action:monitor call_id:32 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_haproxy' not found (7 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_haproxy:1' not found (7 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_haproxy' to the rsc list (8 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=11:126:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_haproxy_monitor_0 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=37, reply=1, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_haproxy action:monitor call_id:37 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header ocf-ns_vrouter: 2016/02/17_18:47:50 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9433) = 0: Resource temporarily unavailable (11) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__public_monitor_0:9436 - exited with rc=7 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__public_monitor_0:9436:stderr [ -- empty -- ] Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__public_monitor_0:9436:stdout [ -- empty -- ] Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__public action:monitor call_id:27 pid:9436 exit-code:7 exec-time:19ms queue-time:0ms Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9444) = 0: Resource temporarily unavailable (11) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9446) = 0: Resource temporarily unavailable (11) Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource vip__public after monitor op complete (interval=0) Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation vip__public_monitor_0: not running (node=node-3.test.domain.local, call=27, rc=7, cib-update=16, confirmed=true) Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'vip__public' with monitor op Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/16) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.748 2 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.749 (null) Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=749 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.750 2 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.751 (null) Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=751 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_vrouter']: word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_rabbitmq-server' not found (9 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_rabbitmq-server:0' not found (9 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_rabbitmq-server' to the rsc list (10 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=13:126:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_rabbitmq-server_monitor_0 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=47, reply=1, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_rabbitmq-server action:monitor call_id:47 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.751 2 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.752 (null) Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=752 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_vrouter']: word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700, cpd=0x7f08d2c5cf04 Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f08d2c5a700 Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5690-9647-26-header Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5690-9647-26-header Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5690-9647-26-header Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-9647-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-9647-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5690-9647-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5690-9647-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5690-9647-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-9647-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [9647] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-9647-26-header Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-9647-26-header Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-9647-26-header Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:47:50 [9647] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-9647-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-9647-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-9647-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-9647-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-9647-26-header ocf-ns_haproxy: 2016/02/17_18:47:50 DEBUG: p_haproxy: get_ns(): added netns haproxy and set up lo ocf-ns_haproxy: 2016/02/17_18:47:50 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:47:50 INFO: haproxy daemon is not running Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9444) = 0: Resource temporarily unavailable (11) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_0:9446 - exited with rc=7 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_0:9446:stderr [ -- empty -- ] Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_0:9446:stdout [ -- empty -- ] Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:37 pid:9446 exit-code:7 exec-time:208ms queue-time:0ms Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_haproxy after monitor op complete (interval=0) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9467) = 0: Resource temporarily unavailable (11) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9546) = 0: Resource temporarily unavailable (11) Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_haproxy_meta-data_0 process 9842 exited with rc=0 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: get_rsc_restart_list: Attr dummy is not reloadable Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/18) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_haproxy_monitor_0: not running (node=node-3.test.domain.local, call=37, rc=7, cib-update=18, confirmed=true) Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_haproxy' with monitor op Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.752 2 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.753 (null) Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=753 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/18, version=0.155.753) Feb 17 18:47:50 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.753 to 0.155.752 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_dns' not found (10 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_dns:0' not found (10 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_dns' to the rsc list (11 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=14:126:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_dns_monitor_0 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=52, reply=1, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_dns action:monitor call_id:52 ocf-ns_dns: 2016/02/17_18:47:50 INFO: dnsmasq daemon is not running Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9444) = 0: Resource temporarily unavailable (11) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9467) = 0: Resource temporarily unavailable (11) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9546) = 0: Resource temporarily unavailable (11) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_0:9895 - exited with rc=7 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_0:9895:stderr [ -- empty -- ] Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_0:9895:stdout [ -- empty -- ] Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:52 pid:9895 exit-code:7 exec-time:19ms queue-time:0ms Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_dns after monitor op complete (interval=0) lrmd: 2016/02/17_18:47:50 DEBUG: p_rabbitmq-server: monitor: action start. Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_dns_meta-data_0 process 9945 exited with rc=0 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_dns_monitor_0: not running (node=node-3.test.domain.local, call=52, rc=7, cib-update=19, confirmed=true) Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_dns' with monitor op Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/19) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.753 2 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.754 (null) Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=754 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/19, version=0.155.754) Feb 17 18:47:50 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.754 to 0.155.753 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_0:9444 - exited with rc=7 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_0:9444:stderr [ -- empty -- ] Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_0:9444:stdout [ vrouter ] Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:32 pid:9444 exit-code:7 exec-time:295ms queue-time:0ms Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9467) = 0: Resource temporarily unavailable (11) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9546) = 0: Resource temporarily unavailable (11) Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 lrmd: 2016/02/17_18:47:50 INFO: p_rabbitmq-server: get_monitor(): CHECK LEVEL IS: 0 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_heat-engine' not found (11 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_heat-engine:0' not found (11 active resources) Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_heat-engine' to the rsc list (12 active resources) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=15:126:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_heat-engine_monitor_0 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=57, reply=1, notify=0, exit=4202464 Feb 17 18:47:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_heat-engine action:monitor call_id:57 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_conntrackd after monitor op complete (interval=0) Feb 17 18:47:50 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_conntrackd_meta-data_0 process 10006 exited with rc=0 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/20) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:50 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:50 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.754 2 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.755 (null) Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=755 Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:47:50 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/22, version=0.155.757) Feb 17 18:47:51 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.757 to 0.155.756 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ocf-neutron-l3-agent_meta-data_0 process 10226 exited with rc=0 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: get_rsc_restart_list: Attr dummy is not reloadable Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/23) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-l3-agent_monitor_0: not running (node=node-3.test.domain.local, call=67, rc=7, cib-update=23, confirmed=true) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-l3-agent' with monitor op Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.757 2 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.758 (null) Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=758 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/23, version=0.155.758) Feb 17 18:47:51 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.758 to 0.155.757 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_neutron-dhcp-agent' not found (14 active resources) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_neutron-dhcp-agent:0' not found (14 active resources) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_neutron-dhcp-agent' to the rsc list (15 active resources) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=18:126:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-dhcp-agent_monitor_0 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=72, reply=1, notify=0, exit=4202464 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_neutron-dhcp-agent action:monitor call_id:72 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_neutron-metadata-agent' not found (15 active resources) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_neutron-metadata-agent:0' not found (15 active resources) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_neutron-metadata-agent' to the rsc list (16 active resources) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=19:126:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-metadata-agent_monitor_0 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=77, reply=1, notify=0, exit=4202464 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_neutron-metadata-agent action:monitor call_id:77 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header ocf-neutron-dhcp-agent: 2016/02/17_18:47:51 DEBUG: Create pid file dir: /var/run/resource-agents/ocf-neutron-dhcp-agent and chown to neutron ocf-neutron-metadata-agent: 2016/02/17_18:47:51 DEBUG: Create pid file dir: /var/run/resource-agents/ocf-neutron-metadata-agent and chown to neutron ocf-neutron-dhcp-agent: 2016/02/17_18:47:51 WARNING: OpenStack Neutron agent 'neutron-dhcp-agent' not running. Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9467) = 0: Resource temporarily unavailable (11) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9546) = 0: Resource temporarily unavailable (11) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_0:10230 - exited with rc=7 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_0:10230:stderr [ -- empty -- ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_0:10230:stdout [ -- empty -- ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:72 pid:10230 exit-code:7 exec-time:55ms queue-time:0ms Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10242) = 0: Resource temporarily unavailable (11) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-dhcp-agent after monitor op complete (interval=0) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ocf-neutron-dhcp-agent_meta-data_0 process 10285 exited with rc=0 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: get_rsc_restart_list: Attr dummy is not reloadable Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/24) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-dhcp-agent_monitor_0: not running (node=node-3.test.domain.local, call=72, rc=7, cib-update=24, confirmed=true) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-dhcp-agent' with monitor op Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.758 2 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.759 (null) Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=759 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/24, version=0.155.759) Feb 17 18:47:51 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.759 to 0.155.758 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-neutron-metadata-agent: 2016/02/17_18:47:51 WARNING: OpenStack Neutron agent 'neutron-metadata-agent' not running. Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9467) = 0: Resource temporarily unavailable (11) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9546) = 0: Resource temporarily unavailable (11) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_0:10242 - exited with rc=7 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_0:10242:stderr [ -- empty -- ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_monitor_0:10242:stdout [ -- empty -- ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-metadata-agent action:monitor call_id:77 pid:10242 exit-code:7 exec-time:62ms queue-time:1ms Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'sysinfo_node-3.test.domain.local' not found (16 active resources) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'sysinfo_node-3.test.domain.local' to the rsc list (17 active resources) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=20:126:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=sysinfo_node-3.test.domain.local_monitor_0 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=81, reply=1, notify=0, exit=4202464 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:81 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-metadata-agent after monitor op complete (interval=0) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9467) = 0: Resource temporarily unavailable (11) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9546) = 0: Resource temporarily unavailable (11) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_0:10298 - exited with rc=7 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: sysinfo_node-3.test.domain.local_monitor_0:10298:stderr [ rm: cannot remove '/var/run//SysInfo-sysinfo_node-3.test.domain.local': No such file or directory ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_0:10298:stdout [ -- empty -- ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:81 pid:10298 exit-code:7 exec-time:11ms queue-time:0ms Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ocf-neutron-metadata-agent_meta-data_0 process 10299 exited with rc=0 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/25) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.759 2 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.760 (null) Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=760 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/25, version=0.155.760) Feb 17 18:47:51 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.760 to 0.155.759 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-metadata-agent_monitor_0: not running (node=node-3.test.domain.local, call=77, rc=7, cib-update=25, confirmed=true) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-metadata-agent' with monitor op Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource sysinfo_node-3.test.domain.local after monitor op complete (interval=0) Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/26) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation sysinfo_node-3.test.domain.local_monitor_0: not running (node=node-3.test.domain.local, call=81, rc=7, cib-update=26, confirmed=true) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: node-3.test.domain.local-sysinfo_node-3.test.domain.local_monitor_0:81 [ rm: cannot remove '/var/run//SysInfo-sysinfo_node-3.test.domain.local': No such file or directory\n ] Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'sysinfo_node-3.test.domain.local' with monitor op Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.760 2 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.761 (null) Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=761 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/26, version=0.155.761) Feb 17 18:47:51 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.761 to 0.155.760 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'sysinfo_node-2.test.domain.local' not found (17 active resources) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'sysinfo_node-2.test.domain.local' to the rsc list (18 active resources) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=21:126:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=sysinfo_node-2.test.domain.local_monitor_0 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=85, reply=1, notify=0, exit=4202464 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:sysinfo_node-2.test.domain.local action:monitor call_id:85 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9467) = 0: Resource temporarily unavailable (11) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9546) = 0: Resource temporarily unavailable (11) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-2.test.domain.local_monitor_0:10330 - exited with rc=7 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: sysinfo_node-2.test.domain.local_monitor_0:10330:stderr [ rm: cannot remove '/var/run//SysInfo-sysinfo_node-2.test.domain.local': No such file or directory ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-2.test.domain.local_monitor_0:10330:stdout [ -- empty -- ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-2.test.domain.local action:monitor call_id:85 pid:10330 exit-code:7 exec-time:6ms queue-time:0ms Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'ping_vip__public' not found (18 active resources) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'ping_vip__public:0' not found (18 active resources) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'ping_vip__public' to the rsc list (19 active resources) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=22:126:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=ping_vip__public_monitor_0 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=90, reply=1, notify=0, exit=4202464 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:ping_vip__public action:monitor call_id:90 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource sysinfo_node-2.test.domain.local after monitor op complete (interval=0) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation sysinfo_node-2.test.domain.local_monitor_0: not running (node=node-3.test.domain.local, call=85, rc=7, cib-update=27, confirmed=true) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: node-3.test.domain.local-sysinfo_node-2.test.domain.local_monitor_0:85 [ rm: cannot remove '/var/run//SysInfo-sysinfo_node-2.test.domain.local': No such file or directory\n ] Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'sysinfo_node-2.test.domain.local' with monitor op Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/27) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9467) = 0: Resource temporarily unavailable (11) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9546) = 0: Resource temporarily unavailable (11) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_0:10337 - exited with rc=7 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_0:10337:stderr [ -- empty -- ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_0:10337:stdout [ -- empty -- ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:ping_vip__public action:monitor call_id:90 pid:10337 exit-code:7 exec-time:4ms queue-time:0ms Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource ping_vip__public after monitor op complete (interval=0) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.761 2 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.762 (null) Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=762 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/27, version=0.155.762) Feb 17 18:47:51 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.762 to 0.155.761 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ping_meta-data_0 process 10345 exited with rc=0 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation ping_vip__public_monitor_0: not running (node=node-3.test.domain.local, call=90, rc=7, cib-update=28, confirmed=true) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'ping_vip__public' with monitor op Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/28) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ffa0 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.762 2 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.763 (null) Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=763 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/28, version=0.155.763) Feb 17 18:47:51 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.763 to 0.155.762 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ffa0 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_ntp' not found (19 active resources) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_get_rsc_info: Resource 'p_ntp:0' not found (19 active resources) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: info: process_lrmd_rsc_register: Added 'p_ntp' to the rsc list (20 active resources) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_register operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=1, notify=1, exit=4202464 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_info operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=0, reply=0, notify=0, exit=4202464 Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=23:126:7:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_ntp_monitor_0 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=95, reply=1, notify=0, exit=4202464 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_ntp action:monitor call_id:95 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header ocf-ns_ntp: 2016/02/17_18:47:51 INFO: ntp daemon is not running Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9467) = 0: Resource temporarily unavailable (11) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9546) = 0: Resource temporarily unavailable (11) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_0:10352 - exited with rc=7 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_0:10352:stderr [ -- empty -- ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_0:10352:stdout [ -- empty -- ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:95 pid:10352 exit-code:7 exec-time:14ms queue-time:0ms Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_ntp after monitor op complete (interval=0) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_ntp_meta-data_0 process 10359 exited with rc=0 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/29) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_ntp_monitor_0: not running (node=node-3.test.domain.local, call=95, rc=7, cib-update=29, confirmed=true) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_ntp' with monitor op Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.763 2 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.764 (null) Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=764 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/29, version=0.155.764) Feb 17 18:47:51 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.764 to 0.155.763 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header lrmd: 2016/02/17_18:47:51 INFO: get_status() failed with code 139. Command output: Error: unable to connect to node 'rabbit@messaging-node-3': nodedown DIAGNOSTICS =========== attempted to contact: ['rabbit@messaging-node-3'] rabbit@messaging-node-3: * connected to epmd (port 4369) on messaging-node-3 * epmd reports: node 'rabbit' not running at all other nodes on messaging-node-3: ['rabbitmq-cli-10067'] * suggestion: start the node current node details: - node name: 'rabbitmq-cli-10067@node-3' - home dir: /var/lib/rabbitmq - cookie hash: soeIWU2jk2YNseTyDSlsEA== Segmentation fault (core dumped) lrmd: 2016/02/17_18:47:51 INFO: p_rabbitmq-server: get_monitor(): get_status() returns 7. lrmd: 2016/02/17_18:47:51 INFO: p_rabbitmq-server: get_monitor(): ensuring this slave does not get promoted. lrmd: 2016/02/17_18:47:51 INFO: p_rabbitmq-server: master_score(): Updating master score attribute with 0 Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-10385-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [10385] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700, cpd=0x7f08d2c59ec4 Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f08d2c5a700 Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5690-10385-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-10385-26) Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5690-10385-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-10385-26) state:2 Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5690-10385-26-header Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5690-10385-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5690-10385-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5690-10385-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-10385-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [10385] Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-10385-26-header Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-10385-26-header Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-10385-26-header Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:47:51 [10385] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-10385-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-10385-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-10385-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-10385-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-10385-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x12e40f0 for uid=0 gid=0 pid=10389 id=b480e887-6e8a-4cd2-9819-3206b8ea988a Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-10389-14) Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [10389] Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=10389 id=8b782a6e-41a4-4f05-a569-4e2772837f70 Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-10389-11) Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10389] Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: master-p_rabbitmq-server=0 Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: (null), Stored: (null) Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of master-p_rabbitmq-server is 0 Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_rabbitmq-server (0) Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_rabbitmq-server'] does not exist Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 29: master-p_rabbitmq-server=0 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/29) Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: master-p_rabbitmq-server=0 for node-3.test.domain.local Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: info: main: Update master-p_rabbitmq-server=0 sent via attrd Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-10389-14-header Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-10389-14-header Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-10389-14-header Feb 17 18:47:51 [10389] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-10389-14) Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-10389-14) state:2 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-10389-14-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-10389-14-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-10389-14-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-10389-11) Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-10389-11) state:2 Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-10389-11-header Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-10389-11-header Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-10389-11-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 lrmd: 2016/02/17_18:47:51 DEBUG: p_rabbitmq-server: monitor: role: Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.764 2 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.765 (null) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=765 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/29, version=0.155.765) Feb 17 18:47:51 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.765 to 0.155.764 Feb 17 18:47:51 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 29 for master-p_rabbitmq-server=0 passed Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] lrmd: 2016/02/17_18:47:51 DEBUG: p_rabbitmq-server: monitor: result: 7 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1113, version=0.155.765) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header lrmd: 2016/02/17_18:47:51 DEBUG: p_rabbitmq-server: monitor: action end. Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(9467) = 0: Resource temporarily unavailable (11) Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_monitor_0:9546 - exited with rc=7 Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:9546:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:9546:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:9546:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:9546:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:9546:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_monitor_0:9546:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_monitor_0:9546:stdout [ -- empty -- ] Feb 17 18:47:51 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_rabbitmq-server action:monitor call_id:47 pid:9546 exit-code:7 exec-time:858ms queue-time:0ms Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_rabbitmq-server after monitor op complete (interval=0) Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-10419-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [10419] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700, cpd=0x7f08d2c50344 Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f08d2c5a700 Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5690-10419-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-10419-26) Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5690-10419-26-header Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5690-10419-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-10419-26) state:2 Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5690-10419-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5690-10419-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5690-10419-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-10419-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [10419] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-10419-26-header Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-10419-26-header Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-10419-26-header Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:47:51 [10419] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-10419-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-10419-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-10419-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-10419-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-10419-26-header Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed rabbitmq-server_meta-data_0 process 10403 exited with rc=0 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/30) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_rabbitmq-server_monitor_0: not running (node=node-3.test.domain.local, call=47, rc=7, cib-update=30, confirmed=true) Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: node-3.test.domain.local-p_rabbitmq-server_monitor_0:47 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n ] Feb 17 18:47:51 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_rabbitmq-server' with monitor op Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.765 2 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.766 (null) Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=766 Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:47:51 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/30, version=0.155.766) Feb 17 18:47:51 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.766 to 0.155.765 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:51 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-mysql-wss: 2016/02/17_18:47:55 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 2 retries left Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:56 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:56 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:56 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:56 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:56 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:56 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:56 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:56 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:47:56 [6775] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: d7ca5f80d8d4226276a28f8449c0d817 for 0.155.766 (0x12f9aa0 0) Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:56 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:56 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:56 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:47:56 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:47:56 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:47:56 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:56 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:47:56 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-mysql-wss: 2016/02/17_18:48:00 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 1 retries left Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:02 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:02 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:02 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:02 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:02 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:02 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60150 Feb 17 18:48:02 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:02 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:02 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:02 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:02 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:02 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:02 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:02 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:02 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:02 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.766 2 Feb 17 18:48:02 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.767 (null) Feb 17 18:48:02 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=767 Feb 17 18:48:02 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9053 Feb 17 18:48:02 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1115, version=0.155.767) Feb 17 18:48:02 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:02 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] does not exist Feb 17 18:48:02 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.767 to 0.155.766 Feb 17 18:48:02 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for var_log_free=(null) passed Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60150 Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:02 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header ocf-mysql-wss: 2016/02/17_18:48:05 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:48:05 INFO: MySQL is not running Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_0:9467 - exited with rc=7 Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_0:9467:stderr [ -- empty -- ] Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_0:9467:stdout [ -- empty -- ] Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_mysql action:monitor call_id:42 pid:9467 exit-code:7 exec-time:15047ms queue-time:1ms Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_mysql after monitor op complete (interval=0) Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed mysql-wss_meta-data_0 process 10607 exited with rc=0 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/31) Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_mysql_monitor_0: not running (node=node-3.test.domain.local, call=42, rc=7, cib-update=31, confirmed=true) Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_mysql' with monitor op Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.767 2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.768 (null) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=768 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources: Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/31, version=0.155.768) Feb 17 18:48:05 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.768 to 0.155.767 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crmd: probe_complete=true Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: true, Current: (null), Stored: (null) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of probe_complete is true Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='probe_complete'] does not exist Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 32: probe_complete=true Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/32) Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: attrd_update_delegate: Sent update: probe_complete=true for node-3.test.domain.local Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60190 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60190 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.768 2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.769 (null) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=769 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/32, version=0.155.769) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.769 to 0.155.768 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 32 for probe_complete=true passed Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1117, version=0.155.769) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for sysinfo_node-3.test.domain.local_start_0 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=148:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=sysinfo_node-3.test.domain.local_start_0 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=96, reply=1, notify=0, exit=4202464 Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:sysinfo_node-3.test.domain.local action:start call_id:96 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:05 [10620] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_haproxy_start_0 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=56:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_haproxy_start_0 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=10620 id=92d1e091-644c-4bbe-a8d9-a022234d3d51 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-10620-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10620] Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10620] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10620] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=97, reply=1, notify=0, exit=4202464 Feb 17 18:48:05 [10620] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: (null), Stored: (null) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of arch is x86_64 Feb 17 18:48:05 [10620] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: arch (x86_64) Feb 17 18:48:05 [10620] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='arch'] does not exist Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/34) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 34: arch=x86_64 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-10620-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-10620-11) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-10620-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-10620-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-10620-11-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_haproxy action:start call_id:97 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [10625] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60150 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-30) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5f1c0 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_mysql_start_0 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=62:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_mysql_start_0 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=98, reply=1, notify=0, exit=4202464 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_mysql action:start call_id:98 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.769 2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.770 (null) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=770 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/34, version=0.155.770) Feb 17 18:48:05 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.770 to 0.155.769 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60150 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-30-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-30-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-30) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-30-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-30) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 34 for arch=x86_64 passed Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5f1c0 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=10625 id=9a657e82-5540-4fdb-a083-b787177b2b57 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-10625-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10625] Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-30-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-30-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-30-header Feb 17 18:48:05 [10625] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10625] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10625] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: (null), Stored: (null) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of os is Linux-3.13.0-77-generic Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: os (Linux-3.13.0-77-generic) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [10625] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:48:05 [10625] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5f1c0 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5f1c0 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1119, version=0.155.770) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='os'] does not exist Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 36: os=Linux-3.13.0-77-generic Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-10625-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-10625-11) state:2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-10625-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-10625-11-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-10625-11-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/36) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_dns_start_0 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=100:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_dns_start_0 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=99, reply=1, notify=0, exit=4202464 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_dns action:start call_id:99 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60ce0 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60ce0 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.770 2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.771 (null) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=771 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/36, version=0.155.771) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.771 to 0.155.770 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 36 for os=Linux-3.13.0-77-generic passed Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1121, version=0.155.771) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-ns_haproxy: 2016/02/17_18:48:05 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:48:05 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name Feb 17 18:48:05 [10678] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=10678 id=f17229c3-8017-4d28-8692-a87458a6d2e9 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-10678-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10678] Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10678] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10678] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10678] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3100 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3100, Current: (null), Stored: (null) Feb 17 18:48:05 [10678] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3100 for localhost Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of free_swap is 3100 Feb 17 18:48:05 [10678] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: free_swap (3100) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='free_swap'] does not exist Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 38: free_swap=3100 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-10678-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-10678-11) state:2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-10678-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-10678-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-10678-11-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/38) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ffe0 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-ns_haproxy: 2016/02/17_18:48:05 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ffe0 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.771 2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.772 (null) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=772 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/38, version=0.155.772) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:05 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.772 to 0.155.771 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 38 for free_swap=3100 passed Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-ns_dns: 2016/02/17_18:48:05 INFO: dnsmasq daemon is not running Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1123, version=0.155.772) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [10707] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=10707 id=a70e0a0f-7064-4be4-8172-97aa31d948b4 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-10707-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10707] Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10707] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10707] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10707] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: (null), Stored: (null) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_info is Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_info (Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_info'] does not exist Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 40: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/40) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-ns_haproxy: 2016/02/17_18:48:05 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60130 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [10707] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:48:05 [10707] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-10707-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-10707-11) state:2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-10707-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-10707-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-10707-11-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.772 2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.773 (null) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=773 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/40, version=0.155.773) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.773 to 0.155.772 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 40 for cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz passed Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60130 ocf-ns_haproxy: 2016/02/17_18:48:05 INFO: haproxy daemon is not running Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1125, version=0.155.773) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [10722] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=10722 id=609005e5-9582-4c34-8fa5-251dbdaa3efb Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-10722-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10722] Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10722] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10722] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10722] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: (null), Stored: (null) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_speed is 6999.99 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_speed (6999.99) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [10722] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:48:05 [10722] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_speed'] does not exist Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 42: cpu_speed=6999.99 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-10722-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-10722-11) state:2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-10722-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-10722-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-10722-11-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/42) Feb 17 18:48:05 [10730] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=10730 id=a7f9c24b-6474-4595-acf9-a4e8e9130c78 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-10730-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10730] Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60190 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:05 [10730] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10730] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10730] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: (null), Stored: (null) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_cores is 2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_cores (2) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [10730] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:48:05 [10730] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.773 2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.774 (null) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=774 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60190 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/42, version=0.155.774) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_cores'] does not exist Feb 17 18:48:05 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.774 to 0.155.773 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 44: cpu_cores=2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 42 for cpu_speed=6999.99 passed Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-10730-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-10730-11) state:2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-10730-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-10730-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-10730-11-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/44) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [10744] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60150 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=10744 id=65aa2627-f959-4113-9cf6-383c32a6a83d Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-10744-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10744] Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10744] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10744] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:05 [10744] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.85, Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.85,, Current: (null), Stored: (null) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.85, Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.85,) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [10744] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.85, for localhost Feb 17 18:48:05 [10744] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.774 2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.775 (null) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=775 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/44, version=0.155.775) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.775 to 0.155.774 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] does not exist Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 46: cpu_load=0.85, Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 44 for cpu_cores=2 passed Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-10744-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-10744-11) state:2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-10744-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-10744-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-10744-11-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60150 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/46) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-ns_dns: 2016/02/17_18:48:05 INFO: stdin: is not a tty Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60150 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1127, version=0.155.775) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60150 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-ns_dns: 2016/02/17_18:48:05 INFO: Started dnsmasq daemon. Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10613) = 0: Resource temporarily unavailable (11) Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10622) = 0: Resource temporarily unavailable (11) Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10627) = 0: Resource temporarily unavailable (11) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_start_0:10637 - exited with rc=0 Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_start_0:10637:stderr [ -- empty -- ] Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_start_0:10637:stdout [ -- empty -- ] Feb 17 18:48:05 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_dns action:start call_id:99 pid:10637 exit-code:0 exec-time:116ms queue-time:0ms Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_dns after start op complete (interval=0) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.775 2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.776 (null) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=776 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:05 [10751] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/46, version=0.155.776) Feb 17 18:48:05 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.776 to 0.155.775 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=10751 id=c233bcd9-9260-4bd0-899d-9a3722a1d21f Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-10751-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10751] Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_dns_meta-data_0 process 10761 exited with rc=0 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/32) Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_dns_start_0: ok (node=node-3.test.domain.local, call=99, rc=0, cib-update=32, confirmed=true) Feb 17 18:48:05 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_dns' with start op Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 46 for cpu_load=0.85, passed Feb 17 18:48:05 [10751] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10751] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10751] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10751] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:48:05 [10751] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: (null), Stored: (null) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of ram_total is 3050 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: ram_total (3050) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_total'] does not exist Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/48) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 48: ram_total=3050 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-10751-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-10751-11) state:2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-10751-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-10751-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-10751-11-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:05 [10769] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ffa0 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.776 2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.777 (null) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=777 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_dns']/lrm_rsc_op[@id='p_dns_last_0']: @operation_key=p_dns_start_0, @operation=start, @transition-key=100:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;100:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=99, @rc-code=0, @last-run=1455734885, @last-rc-change=1455734885, @exec-time=116 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/32, version=0.155.777) Feb 17 18:48:05 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.777 to 0.155.776 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=10769 id=48b79774-9625-4640-a62d-9a9a7de94986 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-10769-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10769] Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10769] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10769] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [10769] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=200 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 200, Current: (null), Stored: (null) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of ram_free is 200 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: ram_free (200) Feb 17 18:48:05 [10769] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=200 for localhost Feb 17 18:48:05 [10769] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ffa0 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header ocf-ns_haproxy: 2016/02/17_18:48:05 INFO: stdin: is not a tty [ALERT] 047/184805 (10731) : sendto logger #1 failed: Resource temporarily unavailable (errno=11) [ALERT] 047/184805 (10731) : sendto logger #1 failed: Resource temporarily unavailable (errno=11) [ALERT] 047/184805 (10731) : sendto logger #1 failed: Resource temporarily unavailable (errno=11) [ALERT] 047/184805 (10731) : sendto logger #1 failed: Resource temporarily unavailable (errno=11) [ALERT] 047/184805 (10731) : sendto logger #1 failed: Resource temporarily unavailable (errno=11) [ALERT] 047/184805 (10731) : sendto logger #1 failed: Resource temporarily unavailable (errno=11) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.777 2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.778 (null) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=778 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/48, version=0.155.778) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] does not exist Feb 17 18:48:05 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.778 to 0.155.777 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 50: ram_free=200 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 48 for ram_total=3050 passed Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-10769-11) Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-10769-11) state:2 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-10769-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-10769-11-header Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-10769-11-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/50) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 ocf-ns_haproxy: 2016/02/17_18:48:05 DEBUG: Creating host interface: hapr-host and namespace interface: hapr-ns Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60150 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:05 [10793] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:05 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.778 2 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.779 (null) Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=779 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_heat-engine']/lrm_rsc_op[@id='p_heat-engine_last_0']: @operation_key=p_heat-engine_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=110:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;110:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=341, @last-run=1455734885, @last-rc-change=1455734 Feb 17 18:48:05 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/486, version=0.155.779) Feb 17 18:48:05 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.779 to 0.155.778 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:05 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60150 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=10793 id=526f1636-0a28-4f3e-b566-2cc81ab63012 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-10793-11) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10793] Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:06 [10793] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [10793] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [10793] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11688 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11688, Current: (null), Stored: (null) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of root_free is 11688 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: root_free (11688) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1129, version=0.155.779) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [10793] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11688 for localhost Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [10793] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.779 2 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.780 (null) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=780 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/50, version=0.155.780) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] does not exist Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 52: root_free=11688 Feb 17 18:48:06 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.780 to 0.155.779 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 50 for ram_free=200 passed Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-10793-11) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-10793-11) state:2 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-10793-11-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-10793-11-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-10793-11-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/52) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-ns_haproxy: 2016/02/17_18:48:06 DEBUG: Moving interface: hapr-ns to namespace: haproxy Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.780 2 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.781 (null) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=781 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_dns']/lrm_rsc_op[@id='p_dns_last_0']: @operation_key=p_dns_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=102:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;102:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=340, @last-run=1455734885, @last-rc-change=1455734885, @exec-time=161, @qu Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/487, version=0.155.781) Feb 17 18:48:06 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.781 to 0.155.780 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ffa0 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [10838] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=10838 id=ecb77c4e-cf53-4086-9631-f783a05abda5 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-10838-11) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10838] Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ffa0 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1131, version=0.155.781) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [10838] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [10838] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [10838] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11688 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11688, Current: 11688, Stored: (null) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of root_free is 11688 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: root_free (11688) Feb 17 18:48:06 [10838] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11688 for localhost Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [10838] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.781 2 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.782 (null) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=782 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/52, version=0.155.782) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[13]) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:06 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.782 to 0.155.781 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 54: root_free=11688 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 52 for root_free=11688 passed Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-10838-11) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-10838-11) state:2 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-10838-11-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-10838-11-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-10838-11-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/54) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60150 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1133, version=0.155.782) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60150 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [10874] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=10874 id=e3ec9638-b4fa-4b2e-89e9-1816ac26a351 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-10874-11) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10874] Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/54, version=0.155.782) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 54 for root_free=11688 passed Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [10874] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [10874] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [10874] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9052 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9052, Current: (null), Stored: (null) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9052 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9052) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [10874] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9052 for localhost Feb 17 18:48:06 [10874] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-ns_haproxy: 2016/02/17_18:48:06 DEBUG: Bringing up host interface: hapr-host Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1135, version=0.155.782) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1137, version=0.155.782) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [10920] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1139, version=0.155.782) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] does not exist Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 56: var_log_free=9052 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/56) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.782 2 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.783 (null) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=783 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/56, version=0.155.783) Feb 17 18:48:06 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.783 to 0.155.782 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x970910 for uid=0 gid=0 pid=10920 id=12cb8c0f-1b68-414a-858c-d045fa78b941 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-10920-12) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [10920] Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-10874-11) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-10874-11) state:2 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-10874-11-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-10874-11-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-10874-11-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 56 for var_log_free=9052 passed Feb 17 18:48:06 [10920] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [10920] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [10920] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-ns_haproxy: 2016/02/17_18:48:06 DEBUG: Setting host interface: hapr-host IP to: 240.0.0.1/30 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18422 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18422, Current: (null), Stored: (null) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_lib_mysql_free is 18422 Feb 17 18:48:06 [10920] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18422 for localhost Feb 17 18:48:06 [10920] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_lib_mysql_free (18422) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:10613 - exited with rc=0 Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:10613:stderr [ -- empty -- ] Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:10613:stdout [ arch: x86_64 ] Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:10613:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:10613:stdout [ free_swap: 3100 ] Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:10613:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:10613:stdout [ cpu_speed: 6999.99 ] Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:10613:stdout [ cpu_cores: 2 ] Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:10613:stdout [ cpu_load: 0.85, ] Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:10613:stdout [ ram_total: 3050 ] Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:10613:stdout [ ram_free: 200 ] Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:10613:stdout [ root_free: 11688 ] Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:10613:stdout [ root_free: 11688 ] Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:10613:stdout [ var_log_free: 9052 ] Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_start_0:10613:stdout [ var_lib_mysql_free: 18422 ] Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:start call_id:96 pid:10613 exit-code:0 exec-time:292ms queue-time:0ms Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10622) = 0: Resource temporarily unavailable (11) Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10627) = 0: Resource temporarily unavailable (11) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_lib_mysql_free'] does not exist Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 58: var_lib_mysql_free=18422 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-10920-12) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-10920-12) state:2 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:06 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource sysinfo_node-3.test.domain.local after start op complete (interval=0) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-10920-12-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-10920-12-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-10920-12-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c600c0 Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c600c0 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1141, version=0.155.783) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/58) Feb 17 18:48:06 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed SysInfo_meta-data_0 process 10950 exited with rc=0 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/33) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation sysinfo_node-3.test.domain.local_start_0: ok (node=node-3.test.domain.local, call=96, rc=0, cib-update=33, confirmed=true) Feb 17 18:48:06 [6905] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-sysinfo_node-3.test.domain.local_start_0:96 [ arch: x86_64\nos: Linux-3.13.0-77-generic\nfree_swap: 3100\ncpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz\ncpu_speed: 6999.99\ncpu_cores: 2\ncpu_load: 0.85,\nram_total: 3050\nram_free: 200\nroot_free: 11688\nroot_free: 11688\nvar_log_free: 9052\nvar_lib_mysql_free: 18422\n ] Feb 17 18:48:06 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'sysinfo_node-3.test.domain.local' with start op Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] ocf-ns_haproxy: 2016/02/17_18:48:06 DEBUG: Bringing up the namespace interface: hapr-ns Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1143, version=0.155.783) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.783 2 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.784 (null) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=784 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:06 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.784 to 0.155.783 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/58, version=0.155.784) Feb 17 18:48:06 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 58 for var_lib_mysql_free=18422 passed Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.784 2 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.785 (null) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=785 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='sysinfo_node-3.test.domain.local']/lrm_rsc_op[@id='sysinfo_node-3.test.domain.local_last_0']: @operation_key=sysinfo_node-3.test.domain.local_start_0, @operation=start, @transition-key=148:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;148:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=96, @rc-code=0, @last-run=14557 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/33, version=0.155.785) Feb 17 18:48:06 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.785 to 0.155.784 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-ns_haproxy: 2016/02/17_18:48:06 DEBUG: Setting namespace interface: hapr-ns IP to: 240.0.0.2/30 ocf-ns_haproxy: 2016/02/17_18:48:06 DEBUG: Creating default route inside the namespace to 240.0.0.1 with metric 10000 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.785 2 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.786 (null) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=786 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_haproxy']/lrm_rsc_op[@id='p_haproxy_last_0']: @operation_key=p_haproxy_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=54:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;54:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=338, @last-run=1455734885, @last-rc-change=1455734885, @exec-tim Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/488, version=0.155.786) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.786 to 0.155.785 ocf-ns_haproxy: 2016/02/17_18:48:06 INFO: net.ipv4.conf.hapr-host.rp_filter = 2 ocf-ns_haproxy: 2016/02/17_18:48:06 INFO: net.ipv4.conf.all.rp_filter = 2 ocf-ns_haproxy: 2016/02/17_18:48:06 DEBUG: Adding route on the host system to 10.109.0.0/24: 240.0.0.2 ocf-ns_haproxy: 2016/02/17_18:48:06 DEBUG: Adding route on the host system to 192.168.2.0/24: 240.0.0.2 ocf-ns_haproxy: 2016/02/17_18:48:06 DEBUG: Adding route on the host system to 192.168.1.0/24: 240.0.0.2 ocf-ns_haproxy: 2016/02/17_18:48:06 DEBUG: Adding route on the host system to 192.168.0.0/24: 240.0.0.2 ocf-ns_haproxy: 2016/02/17_18:48:06 DEBUG: Adding route on the host system to 10.109.1.0/24: 240.0.0.2 ocf-ns_haproxy: 2016/02/17_18:48:06 INFO: Started haproxy daemon. Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_start_0:10622 - exited with rc=0 Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_start_0:10622:stderr [ -- empty -- ] Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_start_0:10622:stdout [ -- empty -- ] Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_haproxy action:start call_id:97 pid:10622 exit-code:0 exec-time:977ms queue-time:1ms Feb 17 18:48:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10627) = 0: Resource temporarily unavailable (11) Feb 17 18:48:06 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_haproxy after start op complete (interval=0) Feb 17 18:48:06 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_haproxy_meta-data_0 process 11342 exited with rc=0 Feb 17 18:48:06 [6905] node-3.test.domain.local crmd: debug: get_rsc_restart_list: Attr dummy is not reloadable Feb 17 18:48:06 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_haproxy_start_0: ok (node=node-3.test.domain.local, call=97, rc=0, cib-update=34, confirmed=true) Feb 17 18:48:06 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_haproxy' with start op Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/34) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.786 2 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.787 (null) Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=787 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_haproxy']/lrm_rsc_op[@id='p_haproxy_last_0']: @operation_key=p_haproxy_start_0, @operation=start, @transition-key=56:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;56:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=97, @rc-code=0, @last-run=1455734885, @last-rc-change=1455734885, @exec-time=977, @queue-time=1 Feb 17 18:48:06 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/34, version=0.155.787) Feb 17 18:48:06 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.787 to 0.155.786 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:06 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-mysql-wss: 2016/02/17_18:48:10 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:48:10 INFO: MySQL is not running ocf-mysql-wss: 2016/02/17_18:48:10 INFO: Create socket dir: /var/run/mysqld and chown to mysql:mysql ocf-mysql-wss: 2016/02/17_18:48:10 INFO: Create PID dir: /var/run/resource-agents/mysql-wss and chown to mysql:mysql Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:11 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:11 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:11 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:11 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:11 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:11 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:11 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:11 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:11 [6775] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: 49ff06e6c30ddb7f86d2e91250e31a3b for 0.155.787 (0x12f9aa0 0) Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:11 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:11 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:11 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:11 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:11 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:11 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:11 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:11 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:13 [6905] node-3.test.domain.local crmd: debug: throttle_cib_load: Init 16 + 6 ticks at 1455734893 (100 tps) Feb 17 18:48:13 [6905] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 2.720000 (full: 2.72 0.85 0.30 3/466 12001) Feb 17 18:48:13 [6905] node-3.test.domain.local crmd: info: throttle_handle_load: Moderate CPU load detected: 2.720000 Feb 17 18:48:13 [6905] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:48:13 [6905] node-3.test.domain.local crmd: debug: throttle_timer_cb: New throttle mode: 0010 (was ffffffff) Feb 17 18:48:13 [6905] node-3.test.domain.local crmd: info: throttle_send_command: Updated throttle state to 0010 Feb 17 18:48:13 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:13 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:13 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:14 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:14 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:14 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:14 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:14 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:14 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:14 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:14 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:14 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:14 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:14 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:14 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:14 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:14 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:14 [6905] node-3.test.domain.local crmd: debug: throttle_update: Host node-3.test.domain.local supports a maximum of 4 jobs and throttle mode 0010. New job limit is 1 Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:14 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60150 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:15 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.787 2 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.788 (null) Feb 17 18:48:15 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=788 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-gtid']: @value=71dae69f-d579-11e5-a980-12eca8fb60f4:21617 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:15 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1145, version=0.155.788) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60150 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] does not exist Feb 17 18:48:15 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.788 to 0.155.787 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for gtid=(null) passed ocf-mysql-wss: 2016/02/17_18:48:15 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:48:15 ERROR: MySQL is not running ocf-mysql-wss: 2016/02/17_18:48:15 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:21617 ocf-mysql-wss: 2016/02/17_18:48:15 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:21617 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x148d880 for uid=0 gid=0 pid=12052 id=e5feef49-196a-4d79-83de-803c97eb012f Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-12052-14) Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12052] Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=12052 id=be2352b9-8299-4242-afdb-1e278af96484 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-12052-11) Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [12052] Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21617 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 71dae69f-d579-11e5-a980-12eca8fb60f4:21617, Current: (null), Stored: (null) Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of gtid is 71dae69f-d579-11e5-a980-12eca8fb60f4:21617 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: gtid (71dae69f-d579-11e5-a980-12eca8fb60f4:21617) Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] does not exist Feb 17 18:48:15 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/61) Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 61: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21617 Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21617 for node-3.test.domain.local Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: info: main: Update gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21617 sent via attrd Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-12052-14) Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-12052-14) state:2 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12052-14-header Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12052-14-header Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12052-14-header Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12052-14-header Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12052-14-header Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12052-14-header Feb 17 18:48:15 [12052] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-12052-11) Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-12052-11) state:2 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-12052-11-header Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-12052-11-header Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-12052-11-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60150 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [12054] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:48:15 [12054] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:48:15 [12054] node-3.test.domain.local crm_node: debug: main: Attempting to process -q command for cluster type: corosync Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:15 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.788 2 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.789 (null) Feb 17 18:48:15 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=789 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:15 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/61, version=0.155.789) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-12054-30) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [12054] Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.789 to 0.155.788 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7f08d2c5e550 Feb 17 18:48:15 [12054] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [12054] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [12054] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:15 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 61 for gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21617 passed Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7f08d2c5e550 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60150 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7f08d2c5e550 Feb 17 18:48:15 [12054] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-12054-30) Feb 17 18:48:15 [12054] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-request-5690-12054-30-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-12054-30) state:2 Feb 17 18:48:15 [12054] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-response-5690-12054-30-header Feb 17 18:48:15 [12054] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-event-5690-12054-30-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:15 [12054] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7f08d2c5e550 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-5690-12054-30-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-5690-12054-30-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-5690-12054-30-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [12063] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:48:15 [12063] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:48:15 [12063] node-3.test.domain.local crm_node: debug: main: Attempting to process -l command for cluster type: corosync Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:15 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1147, version=0.155.789) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:15 [6666] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x20527e0 for uid=0 gid=0 pid=12063 id=2e6a3770-5941-4dfc-8052-96997dfa2332 Feb 17 18:48:15 [6666] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6666-12063-12) Feb 17 18:48:15 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [12063] Feb 17 18:48:15 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:15 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:15 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:15 [12063] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:15 [12063] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:15 [12063] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:15 [12063] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:15 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6666-12063-12) Feb 17 18:48:15 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6666-12063-12) state:2 Feb 17 18:48:15 [6666] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:15 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6666-12063-12-header Feb 17 18:48:15 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6666-12063-12-header Feb 17 18:48:15 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6666-12063-12-header Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x148d880 for uid=0 gid=0 pid=12067 id=9f18ad7f-79fc-4ea5-8965-ce22ee5e7497 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-12067-14) Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12067] Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:15 [12067] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:15 [12067] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:15 [12067] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:15 [12067] node-3.test.domain.local crm_resource: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:15 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: unpack_config: Stop all active resources: false Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: unpack_config: Default stickiness: 0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: determine_op_status: p_mysql_monitor_60000 on node-2.test.domain.local returned 'unknown error' (1) instead of the expected value: 'ok' (0) Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:1 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:1 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-3.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-3.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-3.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-3.test.domain.local to ping_vip__public:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-3.test.domain.local to p_ntp:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:0 Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-12067-14) Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-12067-14) state:2 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12067-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12067-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12067-14-header Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12067-14-header Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12067-14-header Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12067-14-header Feb 17 18:48:16 [12067] node-3.test.domain.local crm_resource: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x148d880 for uid=0 gid=0 pid=12072 id=1a989a53-956a-4870-bec0-e014254cdaba Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-12072-14) Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12072] Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12072] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12072] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12072] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12072] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/configuration/crm_config//cluster_property_set//nvpair[@name='galera_master_timeout'] does not exist Feb 17 18:48:16 [12072] node-3.test.domain.local crm_attribute: info: main: Read galera_master_timeout=300 Feb 17 18:48:16 [12072] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:16 [12072] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:16 [12072] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12072-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-12072-14) Feb 17 18:48:16 [12072] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12072-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-12072-14) state:2 Feb 17 18:48:16 [12072] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12072-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12072-14-header Feb 17 18:48:16 [12072] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12072-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12072-14-header ocf-mysql-wss: 2016/02/17_18:48:16 INFO: Checking if Primary Component Feb 17 18:48:16 [12080] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:48:16 [12080] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:48:16 [12080] node-3.test.domain.local crm_node: debug: main: Attempting to process -p command for cluster type: corosync Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x2073270 for uid=0 gid=0 pid=12080 id=8965af17-29a2-4663-acb9-cc500bb7a2dd Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6666-12080-12) Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [12080] Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:16 [12080] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:16 [12080] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:16 [12080] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:16 [12080] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6666-12080-12) Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6666-12080-12) state:2 Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6666-12080-12-header Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6666-12080-12-header Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6666-12080-12-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x148d880 for uid=0 gid=0 pid=12085 id=2f440f4f-ee14-47e0-9785-7a3fe833d44b Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-12085-14) Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12085] Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: info: main: Read gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21617 Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12085-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-12085-14) Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-12085-14) state:2 Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12085-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12085-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12085-14-header Feb 17 18:48:16 [12085] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12085-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12085-14-header ocf-mysql-wss: 2016/02/17_18:48:16 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:21617 ocf-mysql-wss: 2016/02/17_18:48:16 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:21617 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x148d880 for uid=0 gid=0 pid=12102 id=3b3ed171-a0ec-465d-b111-aa54f543dc83 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-12102-14) Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12102] Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-2.test.domain.local to 2 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='2']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: info: main: Read gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21617 Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12102-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-12102-14) Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12102-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-12102-14) state:2 Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12102-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12102-14-header Feb 17 18:48:16 [12102] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12102-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12102-14-header ocf-mysql-wss: 2016/02/17_18:48:16 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:21617 ocf-mysql-wss: 2016/02/17_18:48:16 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:21617 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x148d880 for uid=0 gid=0 pid=12122 id=4308a203-e721-435b-92b2-0687712bbac0 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-12122-14) Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12122] Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12122-14-header Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12122-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-12122-14) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12122-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-12122-14) state:2 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12122-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12122-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12122-14-header Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: validate_with_relaxng: Creating RNG parser context Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: cib_file_signon: crm_simulate: Opened connection to local file '/var/lib/pacemaker/cib/shadow.12122' Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: cib_file_perform_op_delegate: cib_query on (null) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: unpack_config: Stop all active resources: false Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: unpack_config: Default stickiness: 0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_mysql_monitor_60000 on node-2.test.domain.local returned 'unknown error' (1) instead of the expected value: 'ok' (0) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-3.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-3.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-3.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-3.test.domain.local to ping_vip__public:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-3.test.domain.local to p_ntp:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: apply_system_health: Applying automated node health strategy: migrate-on-red Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-1.test.domain.local has an combined system health of 0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-2.test.domain.local has an combined system health of 0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-3.test.domain.local has an combined system health of 0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-1.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_vrouter [p_vrouter] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_print: vip__management (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter_pub (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_print: vip__public (ocf::fuel:ns_IPaddr2): Stopped Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_conntrackd [p_conntrackd] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_haproxy [p_haproxy] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_mysql [p_mysql] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_dns [p_dns] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_heat-engine [p_heat-engine] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local ] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-3.test.domain.local ] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-plugin-openvswitch-agent [p_neutron-plugin-openvswitch-agent] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-l3-agent [p_neutron-l3-agent] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-dhcp-agent [p_neutron-dhcp-agent] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-metadata-agent [p_neutron-metadata-agent] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-3.test.domain.local (ocf::pacemaker:SysInfo): Started node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-2.test.domain.local (ocf::pacemaker:SysInfo): Started node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_ping_vip__public [ping_vip__public] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_ntp [p_ntp] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__management: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter_pub: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 38 times on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 38 times on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 38 times on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_heat-engine:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:0 has failed 4 times on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource sysinfo_node-1.test.domain.local are unavailable, unclean or shutting down (node-1.test.domain.local: 0, 1000000) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for sysinfo_node-1.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: Resource sysinfo_node-1.test.domain.local cannot run anywhere Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_vrouter:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_vrouter:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_vrouter:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_vrouter:2 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: Resource p_vrouter:2 cannot run anywhere Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_vrouter instances of a possible 3 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: clone_p_haproxy: Rolling back scores from vip__public Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_haproxy:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_haproxy:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_haproxy:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_haproxy:2 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: Resource p_haproxy:2 cannot run anywhere Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_haproxy instances of a possible 3 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__management Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__vrouter_pub Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__vrouter Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource vip__public are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for vip__public Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: Resource vip__public cannot run anywhere Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_conntrackd:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_conntrackd:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: p_conntrackd:2: Rolling back scores from vip__vrouter_pub Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_conntrackd:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_conntrackd:2 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: Resource p_conntrackd:2 cannot run anywhere Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_conntrackd instances of a possible 3 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: master_p_conntrackd: Breaking dependency loop at master_p_conntrackd Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:0 master score: 1000000 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_conntrackd:0 (Stopped node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:2 master score: 0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:1 master score: -1000000 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: master_color: master_p_conntrackd: Promoted 1 instances of a possible 1 to master Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_mysql:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_mysql:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_mysql:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_mysql:2 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: Resource p_mysql:2 cannot run anywhere Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_mysql instances of a possible 3 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_rabbitmq-server:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_rabbitmq-server:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_rabbitmq-server:2 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: Resource p_rabbitmq-server:2 cannot run anywhere Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_rabbitmq-server instances of a possible 3 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:0 master score: 0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_rabbitmq-server:0 (Stopped node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:2 master score: 0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:1 master score: -1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: master_color: master_p_rabbitmq-server: Promoted 1 instances of a possible 1 to master Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:1 with p_vrouter:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_dns:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:0 with p_vrouter:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_dns:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:2 with p_vrouter:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_dns:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_dns:2 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: Resource p_dns:2 cannot run anywhere Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_dns instances of a possible 3 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_heat-engine:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_heat-engine:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_heat-engine:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_heat-engine:2 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: Resource p_heat-engine:2 cannot run anywhere Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_heat-engine instances of a possible 3 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-plugin-openvswitch-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-plugin-openvswitch-agent:2 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-plugin-openvswitch-agent:2 cannot run anywhere Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-plugin-openvswitch-agent instances of a possible 3 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-l3-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-l3-agent:2 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-l3-agent:2 cannot run anywhere Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-l3-agent instances of a possible 3 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-dhcp-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-dhcp-agent:2 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-dhcp-agent:2 cannot run anywhere Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-dhcp-agent instances of a possible 3 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-metadata-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-metadata-agent:2 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-metadata-agent:2 cannot run anywhere Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-metadata-agent instances of a possible 3 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to sysinfo_node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to sysinfo_node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to ping_vip__public:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to ping_vip__public:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource ping_vip__public:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for ping_vip__public:2 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: Resource ping_vip__public:2 cannot run anywhere Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_ping_vip__public instances of a possible 3 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_ntp:0 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_ntp:1 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_ntp:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_ntp:2 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: native_color: Resource p_ntp:2 cannot run anywhere Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_ntp instances of a possible 3 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_conntrackd Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_haproxy:0 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_haproxy:1 on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:0 on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:1 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_rabbitmq-server Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:0 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:1 on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:0 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:1 on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:0 on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:1 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:0 on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:1 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:0 on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:1 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:0 on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:1 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (15s) for sysinfo_node-3.test.domain.local on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:0 on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:1 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:0 on node-3.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:1 on node-2.test.domain.local Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_conntrackd:0 (node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_conntrackd:0 (Stopped -> Master node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_conntrackd:1 (node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:0 (node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:1 (node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_rabbitmq-server:0 (node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_rabbitmq-server:0 (Stopped -> Master node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_rabbitmq-server:1 (node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_heat-engine:1 (node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:0 (node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:1 (node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:0 (node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:1 (node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:0 (node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:1 (node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:0 (node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:1 (node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:0 (node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:1 (node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:0 (node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:1 (node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_haproxy for p_haproxy:0 (aka. p_haproxy) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_haproxy for p_haproxy:1 (aka. p_haproxy) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:0 (aka. p_dns) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:1 (aka. p_dns) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:0 (aka. p_heat-engine) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. p_heat-engine) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. p_heat-engine) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: debug: cib_file_signoff: Signing out of the CIB Service Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: cib_file_signoff: Wrote CIB to /var/lib/pacemaker/cib/shadow.12122 Feb 17 18:48:16 [12122] node-3.test.domain.local crm_simulate: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x148d880 for uid=0 gid=0 pid=12128 id=c15d8c3c-38f8-4955-b22f-da211a3ee957 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-12128-14) Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12128] Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12128-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-12128-14) Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-12128-14) state:2 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12128-14-header Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12128-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12128-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12128-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12128-14-header Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: validate_with_relaxng: Creating RNG parser context Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: cib_file_signon: crm_simulate: Opened connection to local file '/var/lib/pacemaker/cib/shadow.12128' Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: cib_file_perform_op_delegate: cib_query on (null) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: unpack_config: Stop all active resources: false Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: unpack_config: Default stickiness: 0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_mysql_monitor_60000 on node-2.test.domain.local returned 'unknown error' (1) instead of the expected value: 'ok' (0) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-3.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-3.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-3.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-3.test.domain.local to ping_vip__public:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-3.test.domain.local to p_ntp:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: apply_system_health: Applying automated node health strategy: migrate-on-red Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-1.test.domain.local has an combined system health of 0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-2.test.domain.local has an combined system health of 0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-3.test.domain.local has an combined system health of 0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-1.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_vrouter [p_vrouter] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_print: vip__management (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter_pub (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_print: vip__public (ocf::fuel:ns_IPaddr2): Stopped Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_conntrackd [p_conntrackd] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_haproxy [p_haproxy] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_mysql [p_mysql] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_dns [p_dns] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_heat-engine [p_heat-engine] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local ] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-3.test.domain.local ] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-plugin-openvswitch-agent [p_neutron-plugin-openvswitch-agent] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-l3-agent [p_neutron-l3-agent] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-dhcp-agent [p_neutron-dhcp-agent] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-metadata-agent [p_neutron-metadata-agent] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-3.test.domain.local (ocf::pacemaker:SysInfo): Started node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-2.test.domain.local (ocf::pacemaker:SysInfo): Started node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_ping_vip__public [ping_vip__public] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_ntp [p_ntp] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__management: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter_pub: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 38 times on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 38 times on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 38 times on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_heat-engine:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:0 has failed 4 times on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource sysinfo_node-1.test.domain.local are unavailable, unclean or shutting down (node-1.test.domain.local: 0, 1000000) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for sysinfo_node-1.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: Resource sysinfo_node-1.test.domain.local cannot run anywhere Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_vrouter:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_vrouter:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_vrouter:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_vrouter:2 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: Resource p_vrouter:2 cannot run anywhere Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_vrouter instances of a possible 3 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: clone_p_haproxy: Rolling back scores from vip__public Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_haproxy:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_haproxy:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_haproxy:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_haproxy:2 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: Resource p_haproxy:2 cannot run anywhere Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_haproxy instances of a possible 3 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__management Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__vrouter_pub Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__vrouter Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource vip__public are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for vip__public Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: Resource vip__public cannot run anywhere Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_conntrackd:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_conntrackd:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: p_conntrackd:2: Rolling back scores from vip__vrouter_pub Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_conntrackd:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_conntrackd:2 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: Resource p_conntrackd:2 cannot run anywhere Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_conntrackd instances of a possible 3 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: master_p_conntrackd: Breaking dependency loop at master_p_conntrackd Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:0 master score: 1000000 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_conntrackd:0 (Stopped node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:2 master score: 0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:1 master score: -1000000 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: master_color: master_p_conntrackd: Promoted 1 instances of a possible 1 to master Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_mysql:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_mysql:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_mysql:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_mysql:2 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: Resource p_mysql:2 cannot run anywhere Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_mysql instances of a possible 3 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_rabbitmq-server:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_rabbitmq-server:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_rabbitmq-server:2 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: Resource p_rabbitmq-server:2 cannot run anywhere Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_rabbitmq-server instances of a possible 3 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:0 master score: 0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_rabbitmq-server:0 (Stopped node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:2 master score: 0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:1 master score: -1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: master_color: master_p_rabbitmq-server: Promoted 1 instances of a possible 1 to master Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:1 with p_vrouter:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_dns:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:0 with p_vrouter:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_dns:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:2 with p_vrouter:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_dns:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_dns:2 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: Resource p_dns:2 cannot run anywhere Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_dns instances of a possible 3 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_heat-engine:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_heat-engine:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_heat-engine:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_heat-engine:2 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: Resource p_heat-engine:2 cannot run anywhere Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_heat-engine instances of a possible 3 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-plugin-openvswitch-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-plugin-openvswitch-agent:2 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-plugin-openvswitch-agent:2 cannot run anywhere Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-plugin-openvswitch-agent instances of a possible 3 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-l3-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-l3-agent:2 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-l3-agent:2 cannot run anywhere Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-l3-agent instances of a possible 3 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-dhcp-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-dhcp-agent:2 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-dhcp-agent:2 cannot run anywhere Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-dhcp-agent instances of a possible 3 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-metadata-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-metadata-agent:2 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-metadata-agent:2 cannot run anywhere Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-metadata-agent instances of a possible 3 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to sysinfo_node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to sysinfo_node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to ping_vip__public:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to ping_vip__public:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource ping_vip__public:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for ping_vip__public:2 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: Resource ping_vip__public:2 cannot run anywhere Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_ping_vip__public instances of a possible 3 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_ntp:0 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_ntp:1 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_ntp:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_ntp:2 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: native_color: Resource p_ntp:2 cannot run anywhere Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_ntp instances of a possible 3 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_conntrackd Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_haproxy:0 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_haproxy:1 on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:0 on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:1 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_rabbitmq-server Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:0 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:1 on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:0 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:1 on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:0 on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:1 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:0 on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:1 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:0 on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:1 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:0 on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:1 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (15s) for sysinfo_node-3.test.domain.local on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:0 on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:1 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:0 on node-3.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:1 on node-2.test.domain.local Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_conntrackd:0 (node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_conntrackd:0 (Stopped -> Master node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_conntrackd:1 (node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:0 (node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:1 (node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_rabbitmq-server:0 (node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_rabbitmq-server:0 (Stopped -> Master node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_rabbitmq-server:1 (node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_heat-engine:1 (node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:0 (node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:1 (node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:0 (node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:1 (node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:0 (node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:1 (node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:0 (node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:1 (node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:0 (node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:1 (node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:0 (node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:1 (node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_haproxy for p_haproxy:0 (aka. p_haproxy) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_haproxy for p_haproxy:1 (aka. p_haproxy) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:0 (aka. p_dns) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:1 (aka. p_dns) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:0 (aka. p_heat-engine) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. p_heat-engine) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. p_heat-engine) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: debug: cib_file_signoff: Signing out of the CIB Service Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: cib_file_signoff: Wrote CIB to /var/lib/pacemaker/cib/shadow.12128 Feb 17 18:48:16 [12128] node-3.test.domain.local crm_simulate: info: crm_xml_cleanup: Cleaning up memory from libxml2 ocf-mysql-wss: 2016/02/17_18:48:16 INFO: Possible masters: node-3.test.domain.local node-2.test.domain.local ocf-mysql-wss: 2016/02/17_18:48:16 INFO: Choosed master: node-2.test.domain.local Feb 17 18:48:16 [12155] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:48:16 [12155] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:48:16 [12155] node-3.test.domain.local crm_node: debug: main: Attempting to process -q command for cluster type: corosync Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-12155-26) Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [12155] Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:16 [12155] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:16 [12155] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:16 [12155] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7f08d2c5a700 Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7f08d2c5a700 Feb 17 18:48:16 [12155] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:16 [12155] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-request-5690-12155-26-header Feb 17 18:48:16 [12155] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-response-5690-12155-26-header Feb 17 18:48:16 [12155] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-event-5690-12155-26-header Feb 17 18:48:16 [12155] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-12155-26) Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-12155-26) state:2 Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7f08d2c5a700 Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-5690-12155-26-header Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-5690-12155-26-header Feb 17 18:48:16 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-5690-12155-26-header Feb 17 18:48:16 [12164] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:48:16 [12164] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:48:16 [12164] node-3.test.domain.local crm_node: debug: main: Attempting to process -l command for cluster type: corosync Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x2073270 for uid=0 gid=0 pid=12164 id=f1fb9396-0869-43b4-90fb-0f0669c41510 Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6666-12164-12) Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [12164] Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:16 [12164] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:16 [12164] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:16 [12164] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:16 [12164] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6666-12164-12) Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6666-12164-12) state:2 Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6666-12164-12-header Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6666-12164-12-header Feb 17 18:48:16 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6666-12164-12-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x148d880 for uid=0 gid=0 pid=12168 id=7efcfc85-54aa-45f8-af49-cc4fa47db396 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-12168-14) Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12168] Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: unpack_config: Stop all active resources: false Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: unpack_config: Default stickiness: 0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: determine_op_status: p_mysql_monitor_60000 on node-2.test.domain.local returned 'unknown error' (1) instead of the expected value: 'ok' (0) Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:1 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:1 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-3.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-3.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-3.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-3.test.domain.local to ping_vip__public:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-3.test.domain.local to p_ntp:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:0 Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12168-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-12168-14) Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12168-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-12168-14) state:2 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12168-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12168-14-header Feb 17 18:48:16 [12168] node-3.test.domain.local crm_resource: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12168-14-header Feb 17 18:48:16 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12168-14-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60150 Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.789 2 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.790 (null) Feb 17 18:48:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=790 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.50 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1149, version=0.155.790) Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:17 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.790 to 0.155.789 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[10]) Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:17 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/63) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60150 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:17 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/63, version=0.155.790) Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 63 for cpu_load=0.85, passed Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60150 Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.790 2 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.791 (null) Feb 17 18:48:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=791 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9052 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1151, version=0.155.791) Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:17 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.791 to 0.155.790 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[14]) Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:17 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/65) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60150 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:17 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:17 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/65, version=0.155.791) Feb 17 18:48:17 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 65 for var_log_free=9052 passed Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:20 [6818] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:48:20 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:48:20 INFO: net.ipv4.ip_forward = 1 Feb 17 18:48:20 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(10627) = 0: Resource temporarily unavailable (11) Feb 17 18:48:20 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:12248 - exited with rc=0 Feb 17 18:48:20 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:12248:stderr [ -- empty -- ] Feb 17 18:48:20 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:12248:stdout [ -- empty -- ] Feb 17 18:48:20 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:23 pid:12248 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:22 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:22 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:22 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:22 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:22 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:22 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:22 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:22 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:22 [6775] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: dd8a1ef4cf50c71f170b394ee0c009d2 for 0.155.791 (0x12f9aa0 0) Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:22 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:22 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:22 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:22 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:22 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:22 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:22 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:22 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:24 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:24 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:24 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:24 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:24 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:24 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:24 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:24 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:24 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.791 2 Feb 17 18:48:24 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.792 (null) Feb 17 18:48:24 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=792 Feb 17 18:48:24 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_mysql']/lrm_rsc_op[@id='p_mysql_last_0']: @operation_key=p_mysql_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=64:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;64:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=339, @last-run=1455734885, @last-rc-change=1455734885, @exec-time=1845 Feb 17 18:48:24 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/489, version=0.155.792) Feb 17 18:48:24 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.792 to 0.155.791 Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-mysql-wss: 2016/02/17_18:48:26 INFO: Waiting for master. 290 seconds left Feb 17 18:48:26 [12335] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:48:26 [12335] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:48:26 [12335] node-3.test.domain.local crm_node: debug: main: Attempting to process -p command for cluster type: corosync Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x2073270 for uid=0 gid=0 pid=12335 id=875c9ae2-b2b1-40a8-97c2-070d2a8b2c6b Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6666-12335-12) Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [12335] Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:26 [12335] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:26 [12335] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:26 [12335] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:26 [12335] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6666-12335-12) Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6666-12335-12) state:2 Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6666-12335-12-header Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6666-12335-12-header Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6666-12335-12-header Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x148d880 for uid=0 gid=0 pid=12340 id=bf557fa6-b39c-405d-bab4-059d4a1e1b3c Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-12340-14) Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12340] Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: info: main: Read gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21617 Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-12340-14) Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-12340-14) state:2 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12340-14-header Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12340-14-header Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12340-14-header Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12340-14-header Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12340-14-header Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12340-14-header Feb 17 18:48:26 [12340] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 ocf-mysql-wss: 2016/02/17_18:48:26 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:21617 ocf-mysql-wss: 2016/02/17_18:48:26 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:21617 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x148d880 for uid=0 gid=0 pid=12357 id=6f93d973-4814-4146-b5cd-4535b308e225 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-12357-14) Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12357] Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-2.test.domain.local to 2 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='2']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[20]) Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: info: main: Read gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21617 Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12357-14-header Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12357-14-header Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12357-14-header Feb 17 18:48:26 [12357] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-12357-14) Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-12357-14) state:2 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12357-14-header Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12357-14-header Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12357-14-header ocf-mysql-wss: 2016/02/17_18:48:26 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:21617 ocf-mysql-wss: 2016/02/17_18:48:26 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:21617 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x148d880 for uid=0 gid=0 pid=12377 id=f0394849-02de-4903-a115-aab41bd3ff88 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-12377-14) Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12377] Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12377-14-header Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12377-14-header Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12377-14-header Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: validate_with_relaxng: Creating RNG parser context Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-12377-14) Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-12377-14) state:2 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12377-14-header Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12377-14-header Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12377-14-header Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: cib_file_signon: crm_simulate: Opened connection to local file '/var/lib/pacemaker/cib/shadow.12377' Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: cib_file_perform_op_delegate: cib_query on (null) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: unpack_config: Stop all active resources: false Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: unpack_config: Default stickiness: 0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_mysql_monitor_60000 on node-2.test.domain.local returned 'unknown error' (1) instead of the expected value: 'ok' (0) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-3.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-3.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-3.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-3.test.domain.local to ping_vip__public:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-3.test.domain.local to p_ntp:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: apply_system_health: Applying automated node health strategy: migrate-on-red Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-1.test.domain.local has an combined system health of 0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-2.test.domain.local has an combined system health of 0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-3.test.domain.local has an combined system health of 0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-1.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_vrouter [p_vrouter] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_print: vip__management (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter_pub (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_print: vip__public (ocf::fuel:ns_IPaddr2): Stopped Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_conntrackd [p_conntrackd] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_haproxy [p_haproxy] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_mysql [p_mysql] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local ] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-3.test.domain.local ] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_dns [p_dns] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_heat-engine [p_heat-engine] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local ] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-3.test.domain.local ] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-plugin-openvswitch-agent [p_neutron-plugin-openvswitch-agent] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-l3-agent [p_neutron-l3-agent] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-dhcp-agent [p_neutron-dhcp-agent] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-metadata-agent [p_neutron-metadata-agent] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-3.test.domain.local (ocf::pacemaker:SysInfo): Started node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-2.test.domain.local (ocf::pacemaker:SysInfo): Started node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_ping_vip__public [ping_vip__public] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_ntp [p_ntp] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__management: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter_pub: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_mysql:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 38 times on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:1 has failed 38 times on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 38 times on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_heat-engine:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:0 has failed 4 times on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource sysinfo_node-1.test.domain.local are unavailable, unclean or shutting down (node-1.test.domain.local: 0, 1000000) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for sysinfo_node-1.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: Resource sysinfo_node-1.test.domain.local cannot run anywhere Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_vrouter:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_vrouter:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_vrouter:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_vrouter:2 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: Resource p_vrouter:2 cannot run anywhere Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_vrouter instances of a possible 3 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: clone_p_haproxy: Rolling back scores from vip__public Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_haproxy:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_haproxy:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_haproxy:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_haproxy:2 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: Resource p_haproxy:2 cannot run anywhere Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_haproxy instances of a possible 3 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__management Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__vrouter_pub Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__vrouter Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource vip__public are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for vip__public Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: Resource vip__public cannot run anywhere Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_conntrackd:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_conntrackd:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: p_conntrackd:2: Rolling back scores from vip__vrouter_pub Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_conntrackd:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_conntrackd:2 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: Resource p_conntrackd:2 cannot run anywhere Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_conntrackd instances of a possible 3 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: master_p_conntrackd: Breaking dependency loop at master_p_conntrackd Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:0 master score: 1000000 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_conntrackd:0 (Stopped node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:2 master score: 0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:1 master score: -1000000 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: master_color: master_p_conntrackd: Promoted 1 instances of a possible 1 to master Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_mysql:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_mysql:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_mysql:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_mysql:2 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: Resource p_mysql:2 cannot run anywhere Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_mysql instances of a possible 3 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_rabbitmq-server:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_rabbitmq-server:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_rabbitmq-server:2 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: Resource p_rabbitmq-server:2 cannot run anywhere Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_rabbitmq-server instances of a possible 3 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:0 master score: 0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_rabbitmq-server:0 (Stopped node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:2 master score: 0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:1 master score: -1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: master_color: master_p_rabbitmq-server: Promoted 1 instances of a possible 1 to master Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:1 with p_vrouter:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_dns:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:0 with p_vrouter:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_dns:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:2 with p_vrouter:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_dns:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_dns:2 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: Resource p_dns:2 cannot run anywhere Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_dns instances of a possible 3 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_heat-engine:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_heat-engine:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_heat-engine:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_heat-engine:2 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: Resource p_heat-engine:2 cannot run anywhere Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_heat-engine instances of a possible 3 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-plugin-openvswitch-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-plugin-openvswitch-agent:2 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-plugin-openvswitch-agent:2 cannot run anywhere Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-plugin-openvswitch-agent instances of a possible 3 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-l3-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-l3-agent:2 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-l3-agent:2 cannot run anywhere Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-l3-agent instances of a possible 3 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-dhcp-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-dhcp-agent:2 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-dhcp-agent:2 cannot run anywhere Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-dhcp-agent instances of a possible 3 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-metadata-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-metadata-agent:2 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-metadata-agent:2 cannot run anywhere Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-metadata-agent instances of a possible 3 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to sysinfo_node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to sysinfo_node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to ping_vip__public:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to ping_vip__public:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource ping_vip__public:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for ping_vip__public:2 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: Resource ping_vip__public:2 cannot run anywhere Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_ping_vip__public instances of a possible 3 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_ntp:0 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_ntp:1 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_ntp:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_ntp:2 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: native_color: Resource p_ntp:2 cannot run anywhere Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_ntp instances of a possible 3 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_conntrackd Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_haproxy:0 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_haproxy:1 on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:0 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:1 on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_rabbitmq-server Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:0 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:1 on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:0 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:1 on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:0 on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:1 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:0 on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:1 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:0 on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:1 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:0 on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:1 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (15s) for sysinfo_node-3.test.domain.local on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:0 on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:1 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:0 on node-3.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:1 on node-2.test.domain.local Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_conntrackd:0 (node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_conntrackd:0 (Stopped -> Master node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_conntrackd:1 (node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:1 (node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_rabbitmq-server:0 (node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_rabbitmq-server:0 (Stopped -> Master node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_rabbitmq-server:1 (node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_heat-engine:1 (node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:0 (node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:1 (node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:0 (node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:1 (node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:0 (node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:1 (node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:0 (node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:1 (node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:0 (node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:1 (node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:0 (node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:1 (node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_haproxy for p_haproxy:0 (aka. p_haproxy) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_haproxy for p_haproxy:1 (aka. p_haproxy) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. p_mysql) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. p_mysql) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:0 (aka. p_dns) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:1 (aka. p_dns) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:0 (aka. p_heat-engine) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. p_heat-engine) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. p_heat-engine) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: debug: cib_file_signoff: Signing out of the CIB Service Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: cib_file_signoff: Wrote CIB to /var/lib/pacemaker/cib/shadow.12377 Feb 17 18:48:26 [12377] node-3.test.domain.local crm_simulate: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x148d880 for uid=0 gid=0 pid=12386 id=d66f838e-d8ab-462f-9a84-d640a48f1bca Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-12386-14) Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12386] Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-12386-14) Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-12386-14) state:2 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12386-14-header Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12386-14-header Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12386-14-header Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12386-14-header Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12386-14-header Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12386-14-header Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: validate_with_relaxng: Creating RNG parser context Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: cib_file_signon: crm_simulate: Opened connection to local file '/var/lib/pacemaker/cib/shadow.12386' Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: cib_file_perform_op_delegate: cib_query on (null) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: unpack_config: Stop all active resources: false Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: unpack_config: Default stickiness: 0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_mysql_monitor_60000 on node-2.test.domain.local returned 'unknown error' (1) instead of the expected value: 'ok' (0) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: warning: unpack_rsc_op_failure: Processing failed op monitor for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-3.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-3.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-3.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-3.test.domain.local to ping_vip__public:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_ntp on node-3.test.domain.local to p_ntp:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: apply_system_health: Applying automated node health strategy: migrate-on-red Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-1.test.domain.local has an combined system health of 0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-2.test.domain.local has an combined system health of 0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: apply_system_health: Node node-3.test.domain.local has an combined system health of 0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-1.test.domain.local (ocf::pacemaker:SysInfo): Stopped Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_vrouter [p_vrouter] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_print: vip__management (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter_pub (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_print: vip__vrouter (ocf::fuel:ns_IPaddr2): Started node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_print: vip__public (ocf::fuel:ns_IPaddr2): Stopped Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_conntrackd [p_conntrackd] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_haproxy [p_haproxy] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_haproxy:1 active on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_mysql [p_mysql] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_mysql:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local ] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-3.test.domain.local ] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: clone_print: Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_dns [p_dns] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_dns:1 active on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_heat-engine [p_heat-engine] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_active: Resource p_heat-engine:0 active on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: short_print: Started: [ node-2.test.domain.local ] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-3.test.domain.local ] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-plugin-openvswitch-agent [p_neutron-plugin-openvswitch-agent] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-l3-agent [p_neutron-l3-agent] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-dhcp-agent [p_neutron-dhcp-agent] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_neutron-metadata-agent [p_neutron-metadata-agent] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-3.test.domain.local (ocf::pacemaker:SysInfo): Started node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_print: sysinfo_node-2.test.domain.local (ocf::pacemaker:SysInfo): Started node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_ping_vip__public [ping_vip__public] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: clone_print: Clone Set: clone_p_ntp [p_ntp] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: short_print: Stopped: [ node-2.test.domain.local node-3.test.domain.local ] Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__management: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter_pub: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource vip__vrouter: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_mysql:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:0 has failed 38 times on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_mysql:1 has failed 38 times on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_mysql has failed 38 times on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_mysql can fail 999962 more times on node-2.test.domain.local before being forced off Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_heat-engine:0: preferring current location (node=node-2.test.domain.local, weight=1) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: get_failcount_full: p_neutron-l3-agent:0 has failed 4 times on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: get_failcount_full: clone_p_neutron-l3-agent has failed 4 times on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: common_apply_stickiness: clone_p_neutron-l3-agent can fail 999996 more times on node-2.test.domain.local before being forced off Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_vrouter:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_haproxy:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: common_apply_stickiness: Resource p_dns:1: preferring current location (node=node-3.test.domain.local, weight=1) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource sysinfo_node-1.test.domain.local are unavailable, unclean or shutting down (node-1.test.domain.local: 0, 1000000) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for sysinfo_node-1.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: Resource sysinfo_node-1.test.domain.local cannot run anywhere Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_vrouter:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_vrouter:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_vrouter:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_vrouter:2 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: Resource p_vrouter:2 cannot run anywhere Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_vrouter instances of a possible 3 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: clone_p_haproxy: Rolling back scores from vip__public Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_haproxy:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_haproxy:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_haproxy:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_haproxy:2 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: Resource p_haproxy:2 cannot run anywhere Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_haproxy instances of a possible 3 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__management Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__vrouter_pub Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to vip__vrouter Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource vip__public are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for vip__public Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: Resource vip__public cannot run anywhere Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_conntrackd:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_conntrackd:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: p_conntrackd:2: Rolling back scores from vip__vrouter_pub Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_conntrackd:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_conntrackd:2 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: Resource p_conntrackd:2 cannot run anywhere Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_conntrackd instances of a possible 3 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: rsc_merge_weights: master_p_conntrackd: Breaking dependency loop at master_p_conntrackd Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:0 master score: 1000000 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_conntrackd:0 (Stopped node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:2 master score: 0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: master_color: p_conntrackd:1 master score: -1000000 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: master_color: master_p_conntrackd: Promoted 1 instances of a possible 1 to master Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_mysql:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_mysql:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_mysql:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_mysql:2 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: Resource p_mysql:2 cannot run anywhere Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_mysql instances of a possible 3 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_rabbitmq-server:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_rabbitmq-server:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_rabbitmq-server:2 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: Resource p_rabbitmq-server:2 cannot run anywhere Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 master_p_rabbitmq-server instances of a possible 3 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:0 master score: 0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: master_color: Promoting p_rabbitmq-server:0 (Stopped node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:2 master score: 0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: master_color: p_rabbitmq-server:1 master score: -1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: master_color: master_p_rabbitmq-server: Promoted 1 instances of a possible 1 to master Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:1 with p_vrouter:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_dns:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:0 with p_vrouter:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_dns:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_rsc_colocation_rh: Pairing p_dns:2 with p_vrouter:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_dns:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_dns:2 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: Resource p_dns:2 cannot run anywhere Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_dns instances of a possible 3 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_heat-engine:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_heat-engine:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_heat-engine:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_heat-engine:2 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: Resource p_heat-engine:2 cannot run anywhere Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_heat-engine instances of a possible 3 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-plugin-openvswitch-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-plugin-openvswitch-agent:2 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-plugin-openvswitch-agent:2 cannot run anywhere Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-plugin-openvswitch-agent instances of a possible 3 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-l3-agent:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-l3-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-l3-agent:2 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-l3-agent:2 cannot run anywhere Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-l3-agent instances of a possible 3 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-dhcp-agent:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-dhcp-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-dhcp-agent:2 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-dhcp-agent:2 cannot run anywhere Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-dhcp-agent instances of a possible 3 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_neutron-metadata-agent:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_neutron-metadata-agent:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_neutron-metadata-agent:2 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: Resource p_neutron-metadata-agent:2 cannot run anywhere Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_neutron-metadata-agent instances of a possible 3 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to sysinfo_node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to sysinfo_node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to ping_vip__public:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to ping_vip__public:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource ping_vip__public:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for ping_vip__public:2 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: Resource ping_vip__public:2 cannot run anywhere Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_ping_vip__public instances of a possible 3 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-3.test.domain.local to p_ntp:0 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Assigning node-2.test.domain.local to p_ntp:1 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: All nodes for resource p_ntp:2 are unavailable, unclean or shutting down (node-1.test.domain.local: 0, -1000000) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: native_assign_node: Could not allocate a node for p_ntp:2 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: native_color: Resource p_ntp:2 cannot run anywhere Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: clone_color: Allocated 2 clone_p_ntp instances of a possible 3 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_conntrackd Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_conntrackd:0 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_conntrackd:1 on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_haproxy:0 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_haproxy:1 on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:0 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_mysql:1 on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: master_create_actions: Creating actions for master_p_rabbitmq-server Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (27s) for p_rabbitmq-server:0 on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (30s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (103s) for p_rabbitmq-server:1 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:0 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_dns:1 on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:0 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_heat-engine:1 on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:0 on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-plugin-openvswitch-agent:1 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:0 on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-l3-agent:1 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:0 on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_neutron-dhcp-agent:1 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:0 on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (60s) for p_neutron-metadata-agent:1 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (15s) for sysinfo_node-3.test.domain.local on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:0 on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for ping_vip__public:1 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:0 on node-3.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: RecurringOp: Start recurring monitor (20s) for p_ntp:1 on node-2.test.domain.local Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_conntrackd:0 (node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_conntrackd:0 (Stopped -> Master node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_conntrackd:1 (node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_mysql:1 (node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_rabbitmq-server:0 (node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Promote p_rabbitmq-server:0 (Stopped -> Master node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_rabbitmq-server:1 (node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_heat-engine:1 (node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:0 (node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-plugin-openvswitch-agent:1 (node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:0 (node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-l3-agent:1 (node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:0 (node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-dhcp-agent:1 (node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:0 (node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_neutron-metadata-agent:1 (node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:0 (node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start ping_vip__public:1 (node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:0 (node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: notice: LogActions: Start p_ntp:1 (node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:0 (aka. p_conntrackd) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_conntrackd for p_conntrackd:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_haproxy for p_haproxy:0 (aka. p_haproxy) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_haproxy for p_haproxy:1 (aka. p_haproxy) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:0 (aka. p_mysql) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. p_mysql) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_mysql for p_mysql:1 (aka. p_mysql) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:0 (aka. p_rabbitmq-server) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_rabbitmq-server for p_rabbitmq-server:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:0 (aka. p_dns) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_dns for p_dns:1 (aka. p_dns) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:0 (aka. p_heat-engine) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. p_heat-engine) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_heat-engine for p_heat-engine:1 (aka. p_heat-engine) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:0 (aka. p_neutron-plugin-openvswitch-agent) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-plugin-openvswitch-agent for p_neutron-plugin-openvswitch-agent:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:0 (aka. p_neutron-l3-agent) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-l3-agent for p_neutron-l3-agent:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:0 (aka. p_neutron-dhcp-agent) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-dhcp-agent for p_neutron-dhcp-agent:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:0 (aka. p_neutron-metadata-agent) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_neutron-metadata-agent for p_neutron-metadata-agent:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:0 (aka. ping_vip__public) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name ping_vip__public for ping_vip__public:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:0 (aka. p_ntp) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: action2xml: Using anonymous clone name p_ntp for p_ntp:1 (aka. (null)) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-1.test.domain.local (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:1 (Started node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_vrouter:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__management (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter_pub (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__vrouter (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave vip__public (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_conntrackd:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:1 (Started node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_haproxy:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_mysql:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_rabbitmq-server:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:1 (Started node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_dns:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:0 (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_heat-engine:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-plugin-openvswitch-agent:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-l3-agent:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-dhcp-agent:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_neutron-metadata-agent:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-3.test.domain.local (Started node-3.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave sysinfo_node-2.test.domain.local (Started node-2.test.domain.local) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave ping_vip__public:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: LogActions: Leave p_ntp:2 (Stopped) Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: debug: cib_file_signoff: Signing out of the CIB Service Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: cib_file_signoff: Wrote CIB to /var/lib/pacemaker/cib/shadow.12386 Feb 17 18:48:26 [12386] node-3.test.domain.local crm_simulate: info: crm_xml_cleanup: Cleaning up memory from libxml2 ocf-mysql-wss: 2016/02/17_18:48:26 INFO: Possible masters: node-3.test.domain.local node-2.test.domain.local ocf-mysql-wss: 2016/02/17_18:48:26 INFO: Choosed master: node-2.test.domain.local Feb 17 18:48:26 [12413] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:48:26 [12413] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:48:26 [12413] node-3.test.domain.local crm_node: debug: main: Attempting to process -q command for cluster type: corosync Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-12413-26) Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [12413] Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:26 [12413] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:26 [12413] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:26 [12413] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_gettype got quorum_type request on 0x7f08d2c5a700 Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [QUORUM] message_handler_req_lib_quorum_getquorate got quorate request on 0x7f08d2c5a700 Feb 17 18:48:26 [12413] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:26 [12413] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-request-5690-12413-26-header Feb 17 18:48:26 [12413] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-response-5690-12413-26-header Feb 17 18:48:26 [12413] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-quorum-event-5690-12413-26-header Feb 17 18:48:26 [12413] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-12413-26) Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-12413-26) state:2 Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7f08d2c5a700 Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-5690-12413-26-header Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-5690-12413-26-header Feb 17 18:48:26 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-5690-12413-26-header Feb 17 18:48:26 [12422] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:48:26 [12422] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:48:26 [12422] node-3.test.domain.local crm_node: debug: main: Attempting to process -l command for cluster type: corosync Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: crm_client_new: Connecting 0x2073270 for uid=0 gid=0 pid=12422 id=eab0ec1d-2d36-4530-83ef-ac39740b8542 Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: handle_new_connection: IPC credentials authenticated (6666-12422-12) Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcs_shm_connect: connecting to client [12422] Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:26 [12422] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:26 [12422] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:26 [12422] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:26 [12422] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6666-12422-12) Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6666-12422-12) state:2 Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-response-6666-12422-12-header Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-event-6666-12422-12-header Feb 17 18:48:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-pacemakerd-request-6666-12422-12-header Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x148d880 for uid=0 gid=0 pid=12426 id=c822c3d6-3054-41f7-9481-dc36d12d2098 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-12426-14) Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [12426] Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH timeout: 60000 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: unpack_config: STONITH of failed nodes is disabled Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: unpack_config: Stop all active resources: false Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: unpack_config: Default stickiness: 0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: unpack_config: On loss of CCM Quorum: Stop ALL resources Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-2.test.domain.local is online Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: info: determine_online_status: Node node-3.test.domain.local is online Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-2.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-2.test.domain.local to p_vrouter:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-2.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:0 active on node-2.test.domain.local Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-2.test.domain.local to p_heat-engine:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-2.test.domain.local to ping_vip__public:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-2.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: determine_op_status: p_neutron-l3-agent_monitor_20000 on node-2.test.domain.local returned 'not running' (7) instead of the expected value: 'ok' (0) Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_neutron-l3-agent:0 on node-2.test.domain.local: not running (7) Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-2.test.domain.local to p_mysql:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: determine_op_status: p_mysql_monitor_60000 on node-2.test.domain.local returned 'unknown error' (1) instead of the expected value: 'ok' (0) Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: warning: unpack_rsc_op_failure: Processing failed op monitor for p_mysql:0 on node-2.test.domain.local: unknown error (1) Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-2.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-2.test.domain.local to p_ntp:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-2.test.domain.local to p_dns:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-2.test.domain.local to p_conntrackd:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-2.test.domain.local to p_haproxy:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-2.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-2.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_vrouter on node-3.test.domain.local to p_vrouter:1 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: determine_op_status: p_vrouter_monitor_0 on node-3.test.domain.local returned 'ok' (0) instead of the expected value: 'not running' (7) Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: info: determine_op_status: Operation monitor found resource p_vrouter:1 active on node-3.test.domain.local Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_haproxy on node-3.test.domain.local to p_haproxy:1 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_dns on node-3.test.domain.local to p_dns:1 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_conntrackd on node-3.test.domain.local to p_conntrackd:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_heat-engine on node-3.test.domain.local to p_heat-engine:1 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-plugin-openvswitch-agent on node-3.test.domain.local to p_neutron-plugin-openvswitch-agent:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-l3-agent on node-3.test.domain.local to p_neutron-l3-agent:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-dhcp-agent on node-3.test.domain.local to p_neutron-dhcp-agent:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_neutron-metadata-agent on node-3.test.domain.local to p_neutron-metadata-agent:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed ping_vip__public on node-3.test.domain.local to ping_vip__public:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_ntp on node-3.test.domain.local to p_ntp:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_rabbitmq-server on node-3.test.domain.local to p_rabbitmq-server:0 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: find_anonymous_clone: Internally renamed p_mysql on node-3.test.domain.local to p_mysql:1 Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-12426-14) Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-12426-14) state:2 Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12426-14-header Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12426-14-header Feb 17 18:48:26 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12426-14-header Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-12426-14-header Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-12426-14-header Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-12426-14-header Feb 17 18:48:26 [12426] node-3.test.domain.local crm_resource: info: crm_xml_cleanup: Cleaning up memory from libxml2 ocf-mysql-wss: 2016/02/17_18:48:26 INFO: My neighbour is Primary Component ocf-mysql-wss: 2016/02/17_18:48:26 INFO: Starting MySQL Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:29 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:29 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:29 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:29 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:29 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:29 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:29 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:29 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:29 [6775] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: e0f5406cd07af45dd8499ab39226db83 for 0.155.792 (0x12f9aa0 0) Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:29 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:29 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:29 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:29 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:29 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:29 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:29 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-mysql-wss: 2016/02/17_18:48:31 INFO: PIDFile /var/run/resource-agents/mysql-wss/mysql-wss.pid of MySQL server not found. Sleeping for 5 seconds. 0 retries left ocf-mysql-wss: 2016/02/17_18:48:31 INFO: MySQL is not running Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60150 Feb 17 18:48:32 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:32 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:32 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:32 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:32 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.792 2 Feb 17 18:48:32 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.793 (null) Feb 17 18:48:32 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=793 Feb 17 18:48:32 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9051 Feb 17 18:48:32 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1153, version=0.155.793) Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[14]) Feb 17 18:48:32 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.793 to 0.155.792 Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:32 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60150 Feb 17 18:48:32 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/67) Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:32 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:32 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/67, version=0.155.793) Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:32 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 67 for var_log_free=9052 passed Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-mysql-wss: 2016/02/17_18:48:34 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:48:34 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:48:34 INFO: MySQL started ocf-mysql-wss: 2016/02/17_18:48:34 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:48:34 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:48:34 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:21629 ocf-mysql-wss: 2016/02/17_18:48:34 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:21629 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x148e440 for uid=0 gid=0 pid=13670 id=a337690f-77d7-4524-91bd-b63a8eaae535 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-13670-14) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13670] Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=13670 id=354331c0-db85-4af3-a687-2aab6a6a4c8f Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-13670-11) Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13670] Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21629 Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 71dae69f-d579-11e5-a980-12eca8fb60f4:21629, Current: 71dae69f-d579-11e5-a980-12eca8fb60f4:21617, Stored: 71dae69f-d579-11e5-a980-12eca8fb60f4:21617 Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of gtid is 71dae69f-d579-11e5-a980-12eca8fb60f4:21629 Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: gtid (71dae69f-d579-11e5-a980-12eca8fb60f4:21629) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 69: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21629 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/69) Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21629 for node-3.test.domain.local Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: info: main: Update gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21629 sent via attrd Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-13670-14) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-13670-14) state:2 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-13670-14-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-13670-14-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-13670-14-header Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-13670-14-header Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-13670-14-header Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-13670-14-header Feb 17 18:48:34 [13670] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:34 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_start_0:10627 - exited with rc=0 Feb 17 18:48:34 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:10627:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:48:34 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:10627:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:48:34 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:10627:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:48:34 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:10627:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:48:34 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:10627:stderr [ Warning: Using a password on the command line interface can be insecure. ] Feb 17 18:48:34 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_start_0:10627:stderr [ Warning: Using a password on the command line interface can be insecure. ] Feb 17 18:48:34 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_start_0:10627:stdout [ -- empty -- ] Feb 17 18:48:34 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_mysql action:start call_id:98 pid:10627 exit-code:0 exec-time:28807ms queue-time:1ms Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_mysql after start op complete (interval=0) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60150 Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-13670-11) Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-13670-11) state:2 Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-13670-11-header Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-13670-11-header Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-13670-11-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.793 2 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.794 (null) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=794 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-gtid']: @value=71dae69f-d579-11e5-a980-12eca8fb60f4:21629 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed mysql-wss_meta-data_0 process 13671 exited with rc=0 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/69, version=0.155.794) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/35) Feb 17 18:48:34 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.794 to 0.155.793 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_mysql_start_0: ok (node=node-3.test.domain.local, call=98, rc=0, cib-update=35, confirmed=true) Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_mysql_start_0:98 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\nWarning: Using a password on the command line interface can be insecure.\nWarning: Using a password on the command line interface can be insecure.\n ] Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_mysql' with start op Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 69 for gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21629 passed Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60150 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1155, version=0.155.794) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.794 2 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.795 (null) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=795 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_mysql']/lrm_rsc_op[@id='p_mysql_last_0']: @operation_key=p_mysql_start_0, @operation=start, @transition-key=62:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;62:128:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=98, @rc-code=0, @last-run=1455734885, @last-rc-change=1455734885, @exec-time=28807 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/35, version=0.155.795) Feb 17 18:48:34 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.795 to 0.155.794 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=59:129:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_haproxy_monitor_30000 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:34 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=100, reply=1, notify=0, exit=4202464 Feb 17 18:48:34 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_haproxy action:monitor call_id:100 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header ocf-ns_haproxy: 2016/02/17_18:48:34 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:48:34 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.795 2 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.796 (null) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=796 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_heat-engine']: word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.796 2 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.797 (null) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=797 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_haproxy']: word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_conntrackd_start_0 Feb 17 18:48:34 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=28:129:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_start_0 Feb 17 18:48:34 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=101, reply=1, notify=0, exit=4202464 Feb 17 18:48:34 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_conntrackd action:start call_id:101 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.797 2 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.798 (null) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=798 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_haproxy']: word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.798 2 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.799 (null) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=799 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_dns']: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 36:bd:48:6d:ee:07 brd ff:ff:ff:ff:ff:ff ocf-ns_conntrackd: 2016/02/17_18:48:34 INFO: Starting conntrackd Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60150 Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60150 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.799 2 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.800 (null) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=800 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-gtid']: @value=71dae69f-d579-11e5-a980-12eca8fb60f4:21629 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1157, version=0.155.800) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:34 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.800 to 0.155.799 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/71) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/71, version=0.155.800) Feb 17 18:48:34 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 71 for gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21629 passed Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:34 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:34 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.800 2 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.801 (null) Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=801 Feb 17 18:48:34 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_mysql']: mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 36:bd:48:6d:ee:07 brd ff:ff:ff:ff:ff:ff Feb 17 18:48:34 [13753] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:48:34 [13753] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-13753-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [13753] Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700, cpd=0x7f08d2c5ffd4 Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f08d2c5a700 Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5690-13753-26-header Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5690-13753-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-13753-26) Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5690-13753-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-13753-26) state:2 Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5690-13753-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5690-13753-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5690-13753-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-13753-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [13753] Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60150 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.801 2 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.802 (null) Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=802 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_neutron-plugin-openvswitch-agent']/lrm_rsc_op[@id='p_neutron-plugin-openvswitch-agent_last_0']: @operation_key=p_neutron-plugin-openvswitch-agent_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=125:129:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;125:129:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/495, version=0.155.802) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60150 Feb 17 18:48:35 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.802 to 0.155.801 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-13753-26-header Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-13753-26-header Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-13753-26-header Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:48:35 [13753] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-13753-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-13753-26) state:2 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-13753-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-13753-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-13753-26-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1165b70 for uid=0 gid=0 pid=13755 id=992d0ec8-820c-4c06-92ae-2cdcef679860 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-13755-14) Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13755] Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=13755 id=01051613-20d9-4e25-a8bd-460991154b9d Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-13755-11) Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13755] Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: master-p_conntrackd=100 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 100, Current: (null), Stored: (null) Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of master-p_conntrackd is 100 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_conntrackd (100) Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_conntrackd'] does not exist Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: master-p_conntrackd=100 for node-3.test.domain.local Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: info: main: Update master-p_conntrackd=100 sent via attrd Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-13755-14-header Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-13755-14-header Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-13755-14-header Feb 17 18:48:35 [13755] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-13755-14) Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-13755-14) state:2 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-13755-14-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-13755-14-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-13755-14-header Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 73: master-p_conntrackd=100 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-13755-11) Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-13755-11) state:2 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-13755-11-header Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-13755-11-header Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-13755-11-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/73) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ffa0 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-13757-30) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [13757] Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f08d2c5e1d0, cpd=0x7f08d2c5e924 Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.802 2 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.803 (null) Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=803 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/73, version=0.155.803) Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 73 for master-p_conntrackd=100 passed Feb 17 18:48:35 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.803 to 0.155.802 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ffa0 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f08d2c5e1d0 Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5690-13757-30-header Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5690-13757-30-header Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5690-13757-30-header Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-13757-30) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-13757-30) state:2 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f08d2c5e1d0 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5690-13757-30-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5690-13757-30-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5690-13757-30-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-13757-29) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [13757] Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ffe0 Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1159, version=0.155.803) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-13757-29) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-13757-29) state:2 Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-13757-29-header Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-13757-29-header Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-13757-29-header Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:48:35 [13757] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ffe0 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-13757-29-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-13757-29-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-13757-29-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x148e440 for uid=0 gid=0 pid=13759 id=463bbdda-ab2f-4474-a202-98d67e539cd7 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-13759-14) Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [13759] Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=13759 id=a317f11f-6686-405e-af51-5f1927784a3b Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-13759-11) Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13759] Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: master-p_conntrackd=100 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100 Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: master-p_conntrackd=100 for node-3.test.domain.local Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: info: main: Update master-p_conntrackd=100 sent via attrd Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-13759-14-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-13759-14) Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-13759-14) state:2 Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-13759-14-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-13759-14-header Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-13759-14-header Feb 17 18:48:35 [13759] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-13759-14-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-13759-14-header Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-13759-11) Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-13759-11) state:2 Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-13759-11-header Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-13759-11-header Feb 17 18:48:35 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-13759-11-header Feb 17 18:48:35 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:13703 - exited with rc=0 Feb 17 18:48:35 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:13703:stderr [ -- empty -- ] Feb 17 18:48:35 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:13703:stdout [ vrouter ] Feb 17 18:48:35 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:13703:stdout [ mgmt-conntrd ] Feb 17 18:48:35 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:13703:stdout [ vrouter ] Feb 17 18:48:35 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_start_0:13703:stdout [ mgmt-conntrd ] Feb 17 18:48:35 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_conntrackd action:start call_id:101 pid:13703 exit-code:0 exec-time:304ms queue-time:0ms Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_conntrackd after start op complete (interval=0) Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_conntrackd_meta-data_0 process 13766 exited with rc=0 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/37) Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_start_0: ok (node=node-3.test.domain.local, call=101, rc=0, cib-update=37, confirmed=true) Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_conntrackd_start_0:101 [ vrouter\n mgmt-conntrd\nvrouter\n mgmt-conntrd\n ] Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_conntrackd' with start op Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.803 2 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.804 (null) Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=804 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_conntrackd']/lrm_rsc_op[@id='p_conntrackd_last_0']: @operation_key=p_conntrackd_start_0, @operation=start, @transition-key=28:129:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;28:129:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=101, @rc-code=0, @last-run=1455734914, @last-rc-change=1455734914, @exec-time=304 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/37, version=0.155.804) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:35 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.804 to 0.155.803 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:35 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.804 2 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.805 (null) Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=805 Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_conntrackd']/lrm_rsc_op[@id='p_conntrackd_last_0']: @operation_key=p_conntrackd_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=25:129:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;25:129:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=346, @last-run=1455734914, @last-rc-change=1455734914, Feb 17 18:48:35 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/496, version=0.155.805) Feb 17 18:48:35 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.805 to 0.155.804 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=201:129:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_notify_0 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:35 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=102, reply=1, notify=0, exit=4202464 Feb 17 18:48:35 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_conntrackd action:notify call_id:102 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:35 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:13774 - exited with rc=0 Feb 17 18:48:35 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:13774:stderr [ -- empty -- ] Feb 17 18:48:35 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:13774:stdout [ -- empty -- ] Feb 17 18:48:35 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_conntrackd action:notify call_id:102 pid:13774 exit-code:0 exec-time:22ms queue-time:0ms Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_conntrackd after notify op complete (interval=0) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_conntrackd_notify_0 from 201:129:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455734915-8 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_notify_0: ok (node=node-3.test.domain.local, call=102, rc=0, cib-update=0, confirmed=true) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:35 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:35 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:40 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:40 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:40 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:40 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:40 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:40 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:40 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:40 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:40 [6775] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: a5c624377fa073ad84e6d845285e1ebb for 0.155.805 (0x12f9aa0 0) Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:40 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:40 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:40 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:40 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:40 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:40 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:40 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:40 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60150 Feb 17 18:48:41 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:41 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:41 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:41 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:41 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.805 2 Feb 17 18:48:41 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.806 (null) Feb 17 18:48:41 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=806 Feb 17 18:48:41 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-master-p_rabbitmq-server']: @value=1 Feb 17 18:48:41 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1161, version=0.155.806) Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:41 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.806 to 0.155.805 Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_rabbitmq-server'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[2]) Feb 17 18:48:41 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:41 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/75) Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60150 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:41 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:41 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/75, version=0.155.806) Feb 17 18:48:41 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 75 for master-p_rabbitmq-server=0 passed Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:41 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.806 2 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.807 (null) Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=807 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_rabbitmq-server']/lrm_rsc_op[@id='p_rabbitmq-server_last_0']: @operation_key=p_rabbitmq-server_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=77:129:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;77:129:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=347, @last-run=1455734914, @last-rc-chan Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/497, version=0.155.807) Feb 17 18:48:42 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.807 to 0.155.806 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=73:130:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_mysql_monitor_60000 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:42 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=103, reply=1, notify=0, exit=4202464 Feb 17 18:48:42 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_mysql action:monitor call_id:103 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header ocf-mysql-wss: 2016/02/17_18:48:42 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:48:42 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:48:42 ERROR: Not enough arguments [1] to ocf_log. ocf-mysql-wss: 2016/02/17_18:48:42 MYSQL PID FOUND: ocf-mysql-wss: 2016/02/17_18:48:42 INFO: GTID OK: 71dae69f-d579-11e5-a980-12eca8fb60f4:21635 ocf-mysql-wss: 2016/02/17_18:48:42 INFO: Galera GTID: 71dae69f-d579-11e5-a980-12eca8fb60f4:21635 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.807 2 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.808 (null) Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=808 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_neutron-plugin-openvswitch-agent']: Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=13930 id=e2414881-3a09-4e78-b188-e633e47497c3 Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-13930-11) Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13930] Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21635 Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 71dae69f-d579-11e5-a980-12eca8fb60f4:21635, Current: 71dae69f-d579-11e5-a980-12eca8fb60f4:21629, Stored: 71dae69f-d579-11e5-a980-12eca8fb60f4:21629 Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of gtid is 71dae69f-d579-11e5-a980-12eca8fb60f4:21635 Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: gtid (71dae69f-d579-11e5-a980-12eca8fb60f4:21635) Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21635 for node-3.test.domain.local Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: info: main: Update gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21635 sent via attrd Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-13930-14-header Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-13930-14-header Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-13930-14-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:42 [13930] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='gtid'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[16]) Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-13930-14) Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-13930-14) state:2 Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-13930-14-header Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 77: gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21635 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-13930-14-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-13930-14-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/77) Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-13930-11) Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-13930-11) state:2 Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-13930-11-header Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-13930-11-header Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-13930-11-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60000 Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60000 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.808 2 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.809 (null) Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=809 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-gtid']: @value=71dae69f-d579-11e5-a980-12eca8fb60f4:21635 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/77, version=0.155.809) Feb 17 18:48:42 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.809 to 0.155.808 Feb 17 18:48:42 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 77 for gtid=71dae69f-d579-11e5-a980-12eca8fb60f4:21635 passed Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1163, version=0.155.809) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-mysql-wss: 2016/02/17_18:48:42 DEBUG: MySQL monitor succeeded Feb 17 18:48:42 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_60000:13872 - exited with rc=0 Feb 17 18:48:42 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_monitor_60000:13872:stderr [ Warning: Using a password on the command line interface can be insecure. ] Feb 17 18:48:42 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_monitor_60000:13872:stderr [ Warning: Using a password on the command line interface can be insecure. ] Feb 17 18:48:42 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_monitor_60000:13872:stderr [ Warning: Using a password on the command line interface can be insecure. ] Feb 17 18:48:42 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_monitor_60000:13872:stderr [ Warning: Using a password on the command line interface can be insecure. ] Feb 17 18:48:42 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_mysql_monitor_60000:13872:stderr [ Warning: Using a password on the command line interface can be insecure. ] Feb 17 18:48:42 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_mysql_monitor_60000:13872:stdout [ -- empty -- ] Feb 17 18:48:42 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_mysql action:monitor call_id:103 pid:13872 exit-code:0 exec-time:97ms queue-time:0ms Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_mysql after monitor op complete (interval=60000) Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_mysql_monitor_60000: ok (node=node-3.test.domain.local, call=103, rc=0, cib-update=38, confirmed=false) Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_mysql_monitor_60000:103 [ Warning: Using a password on the command line interface can be insecure.\nWarning: Using a password on the command line interface can be insecure.\nWarning: Using a password on the command line interface can be insecure.\nWarning: Using a password on the command line interface can be insecure.\nWarning: Using a password on the command line interface can be insecure. Feb 17 18:48:42 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_mysql' with monitor op Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/38) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:42 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.809 2 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.810 (null) Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=810 Feb 17 18:48:42 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_mysql']: word_size:263168 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.810 2 Feb 17 18:48:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.811 (null) Feb 17 18:48:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=811 Feb 17 18:48:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_neutron-metadata-agent']/lrm_rsc_op[@id='p_neutron-metadata-agent_last_0']: @operation_key=p_neutron-metadata-agent_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=153:130:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;153:130:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=355, @last-run=14 Feb 17 18:48:43 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/500, version=0.155.811) Feb 17 18:48:43 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.811 to 0.155.810 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:43 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.811 2 Feb 17 18:48:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.812 (null) Feb 17 18:48:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=812 Feb 17 18:48:43 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_neutron-dhcp-agent']/lrm_rsc_op[@id='p_neutron-dhcp-agent_last_0']: @operation_key=p_neutron-dhcp-agent_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=145:130:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;145:130:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=354, @last-run=1455734922, @l Feb 17 18:48:43 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/501, version=0.155.812) Feb 17 18:48:43 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.812 to 0.155.811 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.005000 (15 ticks in 30s) Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 1.900000 (full: 1.90 0.83 0.31 2/497 13971) Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: throttle_timer_cb: New throttle mode: 0000 (was 0010) Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: info: throttle_send_command: Updated throttle state to 0000 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:43 [6905] node-3.test.domain.local crmd: debug: throttle_update: Host node-3.test.domain.local supports a maximum of 4 jobs and throttle mode 0000. New job limit is 4 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:43 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.812 2 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.813 (null) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=813 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_neutron-l3-agent']/lrm_rsc_op[@id='p_neutron-l3-agent_last_0']: @operation_key=p_neutron-l3-agent_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=137:130:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;137:130:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=353, @last-run=1455734922, @last-rc Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/502, version=0.155.813) Feb 17 18:48:44 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.813 to 0.155.812 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=114:131:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_dns_monitor_20000 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=104, reply=1, notify=0, exit=4202464 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_dns action:monitor call_id:104 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=164:131:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=105, reply=1, notify=0, exit=4202464 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:105 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [13989] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=13989 id=469967d9-07d5-4516-9b7b-63dde8e2357a Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-13989-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13989] Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [13989] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [13989] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [13989] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:48:44 [13989] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:48:44 [13989] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=214:131:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_notify_0 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=106, reply=1, notify=0, exit=4202464 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-13989-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-13989-11) state:2 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_conntrackd action:notify call_id:106 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-13989-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-13989-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-13989-11-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_heat-engine_start_0 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=121:131:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_heat-engine_start_0 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=107, reply=1, notify=0, exit=4202464 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_heat-engine action:start call_id:107 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [13999] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=13999 id=0ef617a9-5209-4c48-a9df-00f84e7c63bb Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-13999-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [13999] Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [13999] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [13999] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [13999] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:48:44 [13999] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:48:44 [13999] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-13999-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-13999-11) state:2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-13999-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-13999-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-13999-11-header ocf-ns_dns: 2016/02/17_18:48:44 INFO: dnsmasq daemon running Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:13972 - exited with rc=0 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:13972:stderr [ -- empty -- ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:13972:stdout [ -- empty -- ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:104 pid:13972 exit-code:0 exec-time:65ms queue-time:0ms Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_dns after monitor op complete (interval=20000) Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_dns_monitor_20000: ok (node=node-3.test.domain.local, call=104, rc=0, cib-update=39, confirmed=false) Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_dns' with monitor op Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/39) Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13978) = 0: Resource temporarily unavailable (11) Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13990) = 0: Resource temporarily unavailable (11) Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13994) = 0: Resource temporarily unavailable (11) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled ocf-ns_conntrackd: 2016/02/17_18:48:44 INFO: Sending bulk update in pre promote to peers to allow failback Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.813 2 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.814 (null) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=814 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_dns']: word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_neutron-plugin-openvswitch-agent_start_0 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=129:131:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-plugin-openvswitch-agent_start_0 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=108, reply=1, notify=0, exit=4202464 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_neutron-plugin-openvswitch-agent action:start call_id:108 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [14047] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=14047 id=0524d794-b924-4ec8-8090-664421c99006 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-14047-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14047] Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14047] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14047] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14047] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3100 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3100, Current: 3100, Stored: 3100 Feb 17 18:48:44 [14047] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3100 for localhost Feb 17 18:48:44 [14047] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14047-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14047-11) state:2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14047-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14047-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14047-11-header ocf-neutron-ovs-agent: 2016/02/17_18:48:44 WARNING: OpenStack Neutron agent 'neutron-openvswitch-agent' not running. ocf-heat-engine: 2016/02/17_18:48:44 DEBUG: Create pid file: /var/run/resource-agents/heat-engine/heat-engine.pid with content 14048 Feb 17 18:48:44 [14091] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13978) = 0: Resource temporarily unavailable (11) Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:13990 - exited with rc=0 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:13990:stderr [ -- empty -- ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:13990:stdout [ -- empty -- ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_conntrackd action:notify call_id:106 pid:13990 exit-code:0 exec-time:112ms queue-time:0ms Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_conntrackd after notify op complete (interval=0) Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_conntrackd_notify_0 from 214:131:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455734924-10 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_notify_0: ok (node=node-3.test.domain.local, call=106, rc=0, cib-update=0, confirmed=true) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13994) = 0: Resource temporarily unavailable (11) Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14049) = 0: Resource temporarily unavailable (11) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header ocf-neutron-ovs-agent: 2016/02/17_18:48:44 DEBUG: Create pid file: /var/run/resource-agents/ocf-neutron-ovs-agent/ocf-neutron-ovs-agent.pid with content Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=14091 id=875c82a9-d10f-49fb-8053-b1087e0ae9cd Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-14091-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14091] Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [14091] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14091] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14091] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [14091] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:48:44 [14091] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14091-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14091-11) state:2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14091-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14091-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14091-11-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 ocf-heat-engine: 2016/02/17_18:48:44 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [14109] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_neutron-l3-agent_start_0 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=138:131:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-l3-agent_start_0 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=109, reply=1, notify=0, exit=4202464 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_neutron-l3-agent action:start call_id:109 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=14109 id=6336c027-7a4f-4e96-a3b8-293242110279 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-14109-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14109] Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14109] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14109] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14109] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:48:44 [14109] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:48:44 [14109] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14109-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14109-11) state:2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14109-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14109-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14109-11-header Feb 17 18:48:44 [14121] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining ocf-heat-engine: 2016/02/17_18:48:44 INFO: OpenStack Heat Engine started Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13978) = 0: Resource temporarily unavailable (11) Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_start_0:13994 - exited with rc=0 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_start_0:13994:stderr [ -- empty -- ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_start_0:13994:stdout [ -- empty -- ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_heat-engine action:start call_id:107 pid:13994 exit-code:0 exec-time:183ms queue-time:0ms Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_heat-engine after start op complete (interval=0) Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14049) = 0: Resource temporarily unavailable (11) Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14115) = 0: Resource temporarily unavailable (11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=14121 id=151b8759-cbaa-4efa-a312-21ea9e9e2c51 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-14121-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14121] Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14121] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14121] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14121] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:48:44 [14121] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:48:44 [14121] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14121-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14121-11) state:2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14121-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14121-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14121-11-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed heat-engine_meta-data_0 process 14123 exited with rc=0 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_heat-engine_start_0: ok (node=node-3.test.domain.local, call=107, rc=0, cib-update=40, confirmed=true) Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_heat-engine' with start op Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/40) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.814 2 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.815 (null) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=815 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_heat-engine']/lrm_rsc_op[@id='p_heat-engine_last_0']: @operation_key=p_heat-engine_start_0, @operation=start, @transition-key=121:131:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;121:131:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=107, @rc-code=0, @last-run=1455734924, @last-rc-change=1455734924, @exec-time=183 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/40, version=0.155.815) Feb 17 18:48:44 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.815 to 0.155.814 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-neutron-ovs-agent: 2016/02/17_18:48:44 WARNING: OpenStack Neutron agent (neutron-openvswitch-agent) was run, but no PID file found. Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=122:131:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_heat-engine_monitor_20000 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=110, reply=1, notify=0, exit=4202464 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_heat-engine action:monitor call_id:110 Feb 17 18:48:44 [14139] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining ocf-neutron-ovs-agent: 2016/02/17_18:48:44 WARNING: Writing PID='14090' to '/var/run/resource-agents/ocf-neutron-ovs-agent/ocf-neutron-ovs-agent.pid' for 'neutron-openvswitch-agent' worker... Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=14139 id=23a925e0-8581-415a-969a-2e78223f0f94 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-14139-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14139] Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14139] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14139] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14139] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.83, Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.83,, Current: 0.85,, Stored: 0.85, Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.83, Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.83,) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[10]) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/79) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 79: cpu_load=0.83, Feb 17 18:48:44 [14139] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.83, for localhost Feb 17 18:48:44 [14139] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 ocf-neutron-ovs-agent: 2016/02/17_18:48:44 INFO: OpenStack OVS agent (neutron-openvswitch-agent) started Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13978) = 0: Resource temporarily unavailable (11) Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_start_0:14049 - exited with rc=0 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_neutron-plugin-openvswitch-agent_start_0:14049:stderr [ sh: 1: cannot create : Directory nonexistent ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_neutron-plugin-openvswitch-agent_start_0:14049:stderr [ cat: /var/run/resource-agents/ocf-neutron-ovs-agent/ocf-neutron-ovs-agent.pid: No such file or directory ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_start_0:14049:stdout [ -- empty -- ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:start call_id:108 pid:14049 exit-code:0 exec-time:257ms queue-time:0ms Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ffa0 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-plugin-openvswitch-agent after start op complete (interval=0) Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14115) = 0: Resource temporarily unavailable (11) Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14138) = 0: Resource temporarily unavailable (11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14139-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14139-11) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14139-11-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14139-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14139-11-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.815 2 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.816 (null) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=816 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.83, Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/79, version=0.155.816) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ffa0 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:44 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.816 to 0.155.815 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 79 for cpu_load=0.83, passed Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1165, version=0.155.816) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [14171] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.816 2 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.817 (null) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=817 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_neutron-dhcp-agent']: word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ocf-neutron-ovs-agent_meta-data_0 process 14158 exited with rc=0 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-plugin-openvswitch-agent_start_0: ok (node=node-3.test.domain.local, call=108, rc=0, cib-update=41, confirmed=true) Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_neutron-plugin-openvswitch-agent_start_0:108 [ sh: 1: cannot create : Directory nonexistent\ncat: /var/run/resource-agents/ocf-neutron-ovs-agent/ocf-neutron-ovs-agent.pid: No such file or directory\n ] Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-plugin-openvswitch-agent' with start op Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=14171 id=ab6fe1bb-d058-4d24-a560-165e422b5a77 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-14171-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14171] Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.817 2 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.818 (null) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=818 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_neutron-metadata-agent']: word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14195] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14195] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14195] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=250 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 250, Current: 200, Stored: 200 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of ram_free is 250 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: ram_free (250) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[12]) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 81: ram_free=250 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/81) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60000 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [14195] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=250 for localhost Feb 17 18:48:44 [14195] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14195-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14195-11) state:2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14195-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14195-11-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14195-11-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.820 2 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.821 (null) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=821 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-ram_free']: @value=250 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/81, version=0.155.821) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60000 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:44 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.821 to 0.155.820 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 81 for ram_free=250 passed ocf-neutron-l3-agent: 2016/02/17_18:48:44 WARNING: OpenStack Neutron agent 'neutron-l3-agent' not running. Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1167, version=0.155.821) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [14224] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=14224 id=3c12a576-a0d8-4351-8e71-e889b0fd2268 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-14224-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14224] Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14224] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14224] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14224] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11689 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11689, Current: 11688, Stored: 11688 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of root_free is 11689 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: root_free (11689) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[13]) Feb 17 18:48:44 [14224] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11689 for localhost Feb 17 18:48:44 [14224] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/83) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 83: root_free=11689 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14224-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14224-11) state:2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14224-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14224-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14224-11-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.821 2 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.822 (null) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=822 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-root_free']: @value=11689 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/83, version=0.155.822) Feb 17 18:48:44 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.822 to 0.155.821 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 83 for root_free=11689 passed Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header ocf-heat-engine: 2016/02/17_18:48:44 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(13978) = 0: Resource temporarily unavailable (11) Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14115) = 0: Resource temporarily unavailable (11) Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:14138 - exited with rc=0 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:14138:stderr [ -- empty -- ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:14138:stdout [ -- empty -- ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:110 pid:14138 exit-code:0 exec-time:291ms queue-time:0ms Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_heat-engine after monitor op complete (interval=20000) Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_heat-engine_monitor_20000: ok (node=node-3.test.domain.local, call=110, rc=0, cib-update=42, confirmed=false) Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_heat-engine' with monitor op Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/42) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1169, version=0.155.822) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.822 2 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.823 (null) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=823 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_heat-engine']: word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14244] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14244] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14244] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11689 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11689, Current: 11689, Stored: 11689 Feb 17 18:48:44 [14244] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11689 for localhost Feb 17 18:48:44 [14244] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14244-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14244-11) state:2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14244-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14244-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14244-11-header Feb 17 18:48:44 [14258] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=14258 id=5fea822b-8a2e-4afe-8f41-32af72c0b103 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-14258-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14258] Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14258] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14258] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14258] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9050 Feb 17 18:48:44 [14258] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9050 for localhost Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9050, Current: 9052, Stored: 9052 Feb 17 18:48:44 [14258] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9050 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9050) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[14]) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 85: var_log_free=9050 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/85) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14258-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14258-11) state:2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14258-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14258-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14258-11-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.823 2 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.824 (null) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=824 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9050 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/85, version=0.155.824) Feb 17 18:48:44 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.824 to 0.155.823 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 85 for var_log_free=9050 passed Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1171, version=0.155.824) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [14270] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=14270 id=bb6d0371-739a-4c3c-9ee1-31a18b257bd8 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-14270-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14270] Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14270] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14270] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [14270] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18422, Stored: 18422 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_lib_mysql_free is 18424 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_lib_mysql_free (18424) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_lib_mysql_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[15]) Feb 17 18:48:44 [14270] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:48:44 [14270] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/87) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.824 2 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.825 (null) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=825 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_lib_mysql_free']: @value=18424 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/87, version=0.155.825) Feb 17 18:48:44 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.825 to 0.155.824 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13978 - exited with rc=0 Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13978:stderr [ -- empty -- ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13978:stdout [ arch: x86_64 ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13978:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13978:stdout [ free_swap: 3100 ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13978:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13978:stdout [ cpu_speed: 6999.99 ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13978:stdout [ cpu_cores: 2 ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13978:stdout [ cpu_load: 0.83, ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13978:stdout [ ram_total: 3050 ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13978:stdout [ ram_free: 250 ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13978:stdout [ root_free: 11689 ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13978:stdout [ root_free: 11689 ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13978:stdout [ var_log_free: 9050 ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:13978:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:105 pid:13978 exit-code:0 exec-time:747ms queue-time:0ms Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource sysinfo_node-3.test.domain.local after monitor op complete (interval=15000) Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation sysinfo_node-3.test.domain.local_monitor_15000: ok (node=node-3.test.domain.local, call=105, rc=0, cib-update=43, confirmed=false) Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-sysinfo_node-3.test.domain.local_monitor_15000:105 [ arch: x86_64\nos: Linux-3.13.0-77-generic\nfree_swap: 3100\ncpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz\ncpu_speed: 6999.99\ncpu_cores: 2\ncpu_load: 0.83,\nram_total: 3050\nram_free: 250\nroot_free: 11689\nroot_free: 11689\nvar_log_free: 9050\nvar_lib_mysql_free: 18424\n ] Feb 17 18:48:44 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'sysinfo_node-3.test.domain.local' with monitor op Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/43) Feb 17 18:48:44 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14115) = 0: Resource temporarily unavailable (11) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 87: var_lib_mysql_free=18424 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 87 for var_lib_mysql_free=18424 passed Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14270-11) Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14270-11) state:2 Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14270-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14270-11-header Feb 17 18:48:44 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14270-11-header Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:44 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:44 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.825 2 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.826 (null) Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=826 Feb 17 18:48:44 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='sysinfo_node-3.test.domain.local']: word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.826 2 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.827 (null) Feb 17 18:48:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=827 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_neutron-l3-agent']/lrm_rsc_op[@id='p_neutron-l3-agent_last_0']: @operation_key=p_neutron-l3-agent_start_0, @operation=start, @transition-key=138:131:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;138:131:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=109, @rc-code=0, @last-run=1455734924, @last-rc-change=1455734924, @exec- Feb 17 18:48:45 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/44, version=0.155.827) Feb 17 18:48:45 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.827 to 0.155.826 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-l3-agent_start_0: ok (node=node-3.test.domain.local, call=109, rc=0, cib-update=44, confirmed=true) Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_neutron-l3-agent_start_0:109 [ No handlers could be found for logger "oslo_config.cfg"\nsh: 1: cannot create : Directory nonexistent\ncat: /var/run/resource-agents/ocf-neutron-l3-agent/ocf-neutron-l3-agent.pid: No such file or directory\n ] Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-l3-agent' with start op Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=136:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=111, reply=1, notify=0, exit=4202464 Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:111 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=145:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-l3-agent_monitor_20000 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=112, reply=1, notify=0, exit=4202464 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_neutron-l3-agent action:monitor call_id:112 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=217:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_notify_0 Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=113, reply=1, notify=0, exit=4202464 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_conntrackd action:notify call_id:113 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_neutron-dhcp-agent_start_0 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=152:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-dhcp-agent_start_0 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=114, reply=1, notify=0, exit=4202464 Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_neutron-dhcp-agent action:start call_id:114 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.827 2 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.828 (null) Feb 17 18:48:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=828 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_ntp']/lrm_rsc_op[@id='p_ntp_last_0']: @operation_key=p_ntp_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=180:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;180:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=364, @last-run=1455734925, @last-rc-change=1455734925, @exec-time=112 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/508, version=0.155.828) Feb 17 18:48:45 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.828 to 0.155.827 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-ns_conntrackd: 2016/02/17_18:48:45 INFO: Sending bulk update in pre promote to peers to allow failback Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:14328 - exited with rc=0 Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:14328:stderr [ -- empty -- ] Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:14328:stdout [ -- empty -- ] Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:111 pid:14328 exit-code:0 exec-time:183ms queue-time:0ms Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-plugin-openvswitch-agent after monitor op complete (interval=20000) Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-plugin-openvswitch-agent_monitor_20000: ok (node=node-3.test.domain.local, call=111, rc=0, cib-update=45, confirmed=false) Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-plugin-openvswitch-agent' with monitor op Feb 17 18:48:45 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/45) Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14332) = 0: Resource temporarily unavailable (11) Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14339) = 0: Resource temporarily unavailable (11) Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14341) = 0: Resource temporarily unavailable (11) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:45 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.828 2 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.829 (null) Feb 17 18:48:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=829 Feb 17 18:48:45 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_neutron-plugin-openvswitch-agent']: word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_neutron-metadata-agent_start_0 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=160:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-metadata-agent_start_0 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=115, reply=1, notify=0, exit=4202464 Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_neutron-metadata-agent action:start call_id:115 Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14332) = 0: Resource temporarily unavailable (11) Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:14339 - exited with rc=0 Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:14339:stderr [ -- empty -- ] Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:14339:stdout [ -- empty -- ] Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_conntrackd action:notify call_id:113 pid:14339 exit-code:0 exec-time:188ms queue-time:2ms Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_conntrackd after notify op complete (interval=0) Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_conntrackd_notify_0 from 217:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455734925-11 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_notify_0: ok (node=node-3.test.domain.local, call=113, rc=0, cib-update=0, confirmed=true) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14341) = 0: Resource temporarily unavailable (11) Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14424) = 0: Resource temporarily unavailable (11) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:14332 - exited with rc=0 Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:14332:stderr [ -- empty -- ] Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:14332:stdout [ -- empty -- ] Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:112 pid:14332 exit-code:0 exec-time:222ms queue-time:5ms Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14341) = 0: Resource temporarily unavailable (11) Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14424) = 0: Resource temporarily unavailable (11) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_rabbitmq-server_start_0 Feb 17 18:48:45 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=89:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_rabbitmq-server_start_0 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:45 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=116, reply=1, notify=0, exit=4202464 Feb 17 18:48:45 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_rabbitmq-server action:start call_id:116 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-l3-agent after monitor op complete (interval=20000) Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-l3-agent_monitor_20000: ok (node=node-3.test.domain.local, call=112, rc=0, cib-update=46, confirmed=false) Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-l3-agent' with monitor op Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/46) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.830 2 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.831 (null) Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=831 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_neutron-l3-agent']: word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header ocf-neutron-dhcp-agent: 2016/02/17_18:48:46 WARNING: OpenStack Neutron agent 'neutron-dhcp-agent' not running. Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for ping_vip__public_start_0 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=170:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=ping_vip__public_start_0 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=117, reply=1, notify=0, exit=4202464 Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:ping_vip__public action:start call_id:117 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:48:46 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:46 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.831 2 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.832 (null) Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=832 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-master-p_conntrackd']: @value=1000 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1175, version=0.155.832) Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_conntrackd'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[17]) Feb 17 18:48:46 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.832 to 0.155.831 Feb 17 18:48:46 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/89) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.832 2 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.833 (null) Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=833 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='p_conntrackd']/lrm_rsc_op[@id='p_conntrackd_last_0']: @operation_key=p_conntrackd_promote_0, @operation=promote, @transition-key=40:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;40:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=366, @last-run=1455734925, @last-rc-change=1455734925, @exec-time=174, @queue-time=1 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/510, version=0.155.833) Feb 17 18:48:46 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.833 to 0.155.832 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-neutron-metadata-agent: 2016/02/17_18:48:46 WARNING: OpenStack Neutron agent 'neutron-metadata-agent' not running. Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/89, version=0.155.833) Feb 17 18:48:46 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 89 for master-p_conntrackd=100 passed Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster ocf-neutron-metadata-agent: 2016/02/17_18:48:46 DEBUG: Create pid file: /var/run/resource-agents/ocf-neutron-metadata-agent/ocf-neutron-metadata-agent.pid with content Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-14494-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [14494] Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700, cpd=0x7f08d2c5ffd4 Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5690-14494-26-header Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5690-14494-26-header Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5690-14494-26-header Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-14494-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-14494-26) state:2 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5690-14494-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5690-14494-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5690-14494-26-header Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-14494-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [14494] Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-14494-26-header Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-14494-26-header Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-14494-26-header Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:48:46 [14494] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-14494-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-14494-26) state:2 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-14494-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-14494-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-14494-26-header ocf-neutron-metadata-agent: 2016/02/17_18:48:46 WARNING: OpenStack Neutron agent (neutron-metadata-agent) was run, but no PID file found. ocf-neutron-metadata-agent: 2016/02/17_18:48:46 WARNING: Writing PID='14492' to '/var/run/resource-agents/ocf-neutron-metadata-agent/ocf-neutron-metadata-agent.pid' for 'neutron-metadata-agent' worker... ocf-neutron-metadata-agent: 2016/02/17_18:48:46 INFO: OpenStack Neutron Metadata Agent (neutron-metadata-agent) started Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14341) = 0: Resource temporarily unavailable (11) Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_start_0:14424 - exited with rc=0 Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_neutron-metadata-agent_start_0:14424:stderr [ sh: 1: cannot create : Directory nonexistent ] Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_neutron-metadata-agent_start_0:14424:stderr [ cat: /var/run/resource-agents/ocf-neutron-metadata-agent/ocf-neutron-metadata-agent.pid: No such file or directory ] Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-metadata-agent_start_0:14424:stdout [ -- empty -- ] Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_neutron-metadata-agent action:start call_id:115 pid:14424 exit-code:0 exec-time:549ms queue-time:0ms Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14432) = 0: Resource temporarily unavailable (11) Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14439) = 0: Resource temporarily unavailable (11) Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-metadata-agent after start op complete (interval=0) Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ocf-neutron-metadata-agent_meta-data_0 process 14580 exited with rc=0 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/47) Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-metadata-agent_start_0: ok (node=node-3.test.domain.local, call=115, rc=0, cib-update=47, confirmed=true) Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_neutron-metadata-agent_start_0:115 [ sh: 1: cannot create : Directory nonexistent\ncat: /var/run/resource-agents/ocf-neutron-metadata-agent/ocf-neutron-metadata-agent.pid: No such file or directory\n ] Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-metadata-agent' with start op Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:46 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.833 2 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.834 (null) Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=834 Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_neutron-metadata-agent']/lrm_rsc_op[@id='p_neutron-metadata-agent_last_0']: @operation_key=p_neutron-metadata-agent_start_0, @operation=start, @transition-key=160:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;160:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=115, @rc-code=0, @last-run=1455734925, @last-rc-change= Feb 17 18:48:46 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/47, version=0.155.834) Feb 17 18:48:46 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.834 to 0.155.833 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=218:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_notify_0 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=118, reply=1, notify=0, exit=4202464 Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_conntrackd action:notify call_id:118 Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14341) = 0: Resource temporarily unavailable (11) Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14432) = 0: Resource temporarily unavailable (11) Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14439) = 0: Resource temporarily unavailable (11) Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:14599 - exited with rc=0 Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:14599:stderr [ -- empty -- ] Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_notify_0:14599:stdout [ -- empty -- ] Feb 17 18:48:46 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_conntrackd action:notify call_id:118 pid:14599 exit-code:0 exec-time:124ms queue-time:0ms Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_conntrackd after notify op complete (interval=0) Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_conntrackd_notify_0 from 218:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455734926-12 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_notify_0: ok (node=node-3.test.domain.local, call=118, rc=0, cib-update=0, confirmed=true) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:46 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:48:46 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header lrmd: 2016/02/17_18:48:46 INFO: p_rabbitmq-server: start: action begin. Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.834 2 Feb 17 18:48:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.835 (null) Feb 17 18:48:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=835 Feb 17 18:48:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='ping_vip__public']/lrm_rsc_op[@id='ping_vip__public_last_0']: @operation_key=ping_vip__public_start_0, @operation=start, @crm-debug-origin=do_update_resource, @transition-key=172:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;172:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=363, @last-run=1455734925, @last-rc-chang Feb 17 18:48:47 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/crmd/511, version=0.155.835) Feb 17 18:48:47 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.835 to 0.155.834 Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-neutron-dhcp-agent: 2016/02/17_18:48:47 DEBUG: Create pid file: /var/run/resource-agents/ocf-neutron-dhcp-agent/ocf-neutron-dhcp-agent.pid with content ocf-neutron-dhcp-agent: 2016/02/17_18:48:47 WARNING: OpenStack Neutron agent (neutron-dhcp-agent) was run, but no PID file found. ocf-neutron-dhcp-agent: 2016/02/17_18:48:47 WARNING: Writing PID='14782' to '/var/run/resource-agents/ocf-neutron-dhcp-agent/ocf-neutron-dhcp-agent.pid' for 'neutron-dhcp-agent' worker... ocf-neutron-dhcp-agent: 2016/02/17_18:48:47 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded ocf-neutron-dhcp-agent: 2016/02/17_18:48:47 INFO: OpenStack DHCP Agent (neutron-dhcp-agent) started Feb 17 18:48:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_start_0:14341 - exited with rc=0 Feb 17 18:48:47 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_neutron-dhcp-agent_start_0:14341:stderr [ No handlers could be found for logger "oslo_config.cfg" ] Feb 17 18:48:47 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_neutron-dhcp-agent_start_0:14341:stderr [ sh: 1: cannot create : Directory nonexistent ] Feb 17 18:48:47 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_neutron-dhcp-agent_start_0:14341:stderr [ cat: /var/run/resource-agents/ocf-neutron-dhcp-agent/ocf-neutron-dhcp-agent.pid: No such file or directory ] Feb 17 18:48:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_start_0:14341:stdout [ -- empty -- ] Feb 17 18:48:47 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_neutron-dhcp-agent action:start call_id:114 pid:14341 exit-code:0 exec-time:2058ms queue-time:0ms Feb 17 18:48:47 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-dhcp-agent after start op complete (interval=0) Feb 17 18:48:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14432) = 0: Resource temporarily unavailable (11) Feb 17 18:48:47 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14439) = 0: Resource temporarily unavailable (11) Feb 17 18:48:47 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ocf-neutron-dhcp-agent_meta-data_0 process 14813 exited with rc=0 Feb 17 18:48:47 [6905] node-3.test.domain.local crmd: debug: get_rsc_restart_list: Attr dummy is not reloadable Feb 17 18:48:47 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/48) Feb 17 18:48:47 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-dhcp-agent_start_0: ok (node=node-3.test.domain.local, call=114, rc=0, cib-update=48, confirmed=true) Feb 17 18:48:47 [6905] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_neutron-dhcp-agent_start_0:114 [ No handlers could be found for logger "oslo_config.cfg"\nsh: 1: cannot create : Directory nonexistent\ncat: /var/run/resource-agents/ocf-neutron-dhcp-agent/ocf-neutron-dhcp-agent.pid: No such file or directory\n ] Feb 17 18:48:47 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-dhcp-agent' with start op Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:47 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.835 2 Feb 17 18:48:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.836 (null) Feb 17 18:48:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=836 Feb 17 18:48:47 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_neutron-dhcp-agent']/lrm_rsc_op[@id='p_neutron-dhcp-agent_last_0']: @operation_key=p_neutron-dhcp-agent_start_0, @operation=start, @transition-key=152:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;152:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=114, @rc-code=0, @last-run=1455734925, @last-rc-change=1455734925, Feb 17 18:48:47 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/48, version=0.155.836) Feb 17 18:48:47 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.836 to 0.155.835 Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:47 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header lrmd: 2016/02/17_18:48:48 INFO: get_status() failed with code 2. Command output: Error: unable to connect to node 'rabbit@messaging-node-3': nodedown DIAGNOSTICS =========== attempted to contact: ['rabbit@messaging-node-3'] rabbit@messaging-node-3: * connected to epmd (port 4369) on messaging-node-3 * epmd reports: node 'rabbit' not running at all other nodes on messaging-node-3: ['rabbitmq-cli-14677'] * suggestion: start the node current node details: - node name: 'rabbitmq-cli-14677@node-3' - home dir: /var/lib/rabbitmq - cookie hash: soeIWU2jk2YNseTyDSlsEA== Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1139380 for uid=0 gid=0 pid=14837 id=e5a8889e-142e-45f3-b5e6-9c94ef55d8e0 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-14837-14) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [14837] Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=14837 id=61e5503e-86f4-4e3e-927f-bb73f3df0eee Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-14837-11) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14837] Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_list_channels_timeouts=0 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: (null), Stored: (null) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of rabbit_list_channels_timeouts is 0 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: rabbit_list_channels_timeouts (0) Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_list_channels_timeouts=0 for node-3.test.domain.local Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: info: main: Update rabbit_list_channels_timeouts=0 sent via attrd Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-14837-14-header Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-14837-14-header Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-14837-14-header Feb 17 18:48:48 [14837] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit_list_channels_timeouts'] does not exist Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-14837-14) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-14837-14) state:2 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-14837-14-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-14837-14-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-14837-14-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 91: rabbit_list_channels_timeouts=0 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14837-11) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14837-11) state:2 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14837-11-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14837-11-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14837-11-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/91) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.836 2 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.837 (null) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=837 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/91, version=0.155.837) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1139380 for uid=0 gid=0 pid=14842 id=8fca6a88-e138-4454-8790-87f597605128 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-14842-14) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [14842] Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.837 to 0.155.836 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 91 for rabbit_list_channels_timeouts=0 passed Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1177, version=0.155.837) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=14842 id=f93f26e2-ada9-4520-a211-82b0cf8a1970 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-14842-11) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14842] Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_get_alarms_timeouts=0 Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_get_alarms_timeouts=0 for node-3.test.domain.local Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: (null), Stored: (null) Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: info: main: Update rabbit_get_alarms_timeouts=0 sent via attrd Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of rabbit_get_alarms_timeouts is 0 Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: rabbit_get_alarms_timeouts (0) Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-14842-14-header Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-14842-14-header Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-14842-14-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [14842] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit_get_alarms_timeouts'] does not exist Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-14842-14) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-14842-14) state:2 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-14842-14-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-14842-14-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-14842-14-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 93: rabbit_get_alarms_timeouts=0 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14842-11) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14842-11) state:2 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14842-11-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14842-11-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14842-11-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/93) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.837 2 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.838 (null) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=838 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.51 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1179, version=0.155.838) Feb 17 18:48:48 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.838 to 0.155.837 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.838 2 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.839 (null) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=839 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/93, version=0.155.839) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1139380 for uid=0 gid=0 pid=14846 id=e463b746-cbce-46bb-be03-4235a385c245 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-14846-14) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [14846] Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[10]) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 93 for rabbit_get_alarms_timeouts=0 passed Feb 17 18:48:48 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.839 to 0.155.838 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/95) Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:48 [14852] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=14846 id=8e24e310-e360-49eb-a552-9dc2232ec22b Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-14846-11) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14846] Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x9716f0 for uid=0 gid=0 pid=14852 id=60ed09a4-113e-45c3-bb48-cf5197eb3a20 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-14852-12) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14852] Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit_list_queues_timeouts=0 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0, Current: (null), Stored: (null) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of rabbit_list_queues_timeouts is 0 Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit_list_queues_timeouts=0 for node-3.test.domain.local Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: info: main: Update rabbit_list_queues_timeouts=0 sent via attrd Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: rabbit_list_queues_timeouts (0) Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-14846-14-header Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-14846-14-header Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-14846-14-header Feb 17 18:48:48 [14846] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:48 [14852] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [14852] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [14852] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1181, version=0.155.839) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 lrmd: 2016/02/17_18:48:48 INFO: p_rabbitmq-server: start: Deleting start time attribute Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/95, version=0.155.839) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x11389a0 for uid=0 gid=0 pid=14860 id=2b65e8b1-f3db-41f0-a25d-7dc0a978e8d7 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-14860-15) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [14860] Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit_list_queues_timeouts'] does not exist Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 97: rabbit_list_queues_timeouts=0 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 95 for cpu_load=0.83, passed Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-14846-14) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-14846-14) state:2 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-14846-14-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-14846-14-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-14846-14-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/97) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.839 2 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.840 (null) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=840 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9050 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1183, version=0.155.840) Feb 17 18:48:48 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.840 to 0.155.839 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.840 2 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.841 (null) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=841 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/97, version=0.155.841) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[14]) Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.841 to 0.155.840 Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/99) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 97 for rabbit_list_queues_timeouts=0 passed Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x972780 for uid=0 gid=0 pid=14860 id=75547999-6bf5-44de-b25d-8c6480a15264 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-14860-13) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14860] Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14846-11) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14846-11) state:2 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14846-11-header Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14846-11-header Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14846-11-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: pingd=1000 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for pingd Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1000, Current: (null), Stored: (null) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of pingd is 1000 Feb 17 18:48:48 [14852] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: pingd=1000 for localhost Feb 17 18:48:48 [14852] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit-start-time= Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for rabbit-start-time Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14852-12) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14852-12) state:2 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14852-12-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14852-12-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14852-12-header Feb 17 18:48:48 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14432) = 0: Resource temporarily unavailable (11) Feb 17 18:48:48 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_start_0:14439 - exited with rc=0 Feb 17 18:48:48 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_start_0:14439:stderr [ -- empty -- ] Feb 17 18:48:48 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_start_0:14439:stdout [ -- empty -- ] Feb 17 18:48:48 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:ping_vip__public action:start call_id:117 pid:14439 exit-code:0 exec-time:2141ms queue-time:0ms Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit-start-time=(null) for node-3.test.domain.local Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: info: main: Update rabbit-start-time= sent via attrd Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-14860-15-header Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-14860-15-header Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-14860-15-header Feb 17 18:48:48 [14860] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14860-13) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14860-13) state:2 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14860-13-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14860-13-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14860-13-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:48:48 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource ping_vip__public after start op complete (interval=0) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1185, version=0.155.841) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 lrmd: 2016/02/17_18:48:48 INFO: p_rabbitmq-server: start: Deleting master attribute Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/99, version=0.155.841) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-14860-15) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-14860-15) state:2 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-14860-15-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 99 for var_log_free=9050 passed Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-14860-15-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-14860-15-header Feb 17 18:48:48 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ping_meta-data_0 process 14865 exited with rc=0 Feb 17 18:48:48 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation ping_vip__public_start_0: ok (node=node-3.test.domain.local, call=117, rc=0, cib-update=49, confirmed=true) Feb 17 18:48:48 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'ping_vip__public' with start op Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x1139380 for uid=0 gid=0 pid=14877 id=ac6bd15e-73d4-41e3-9d61-1d79c9a5e52b Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-14877-14) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [14877] Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/49) Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=14877 id=541bcc82-b99d-4f63-bd7e-766275da11f1 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-14877-11) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [14877] Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: rabbit-master= Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: info: find_hash_entry: Creating hash entry for rabbit-master Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null) Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: rabbit-master=(null) for node-3.test.domain.local Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: info: main: Update rabbit-master= sent via attrd Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-14877-14-header Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-14877-14-header Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-14877-14-header Feb 17 18:48:48 [14877] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-14877-11) Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-14877-11) state:2 Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-14877-11-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-14877-11-header Feb 17 18:48:48 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-14877-11-header lrmd: 2016/02/17_18:48:48 INFO: p_rabbitmq-server: start: RMQ going to start. lrmd: 2016/02/17_18:48:48 INFO: p_rabbitmq-server: start_rmq_server_app(): begin. Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.841 2 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.842 (null) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=842 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='ping_vip__public']/lrm_rsc_op[@id='ping_vip__public_last_0']: @operation_key=ping_vip__public_start_0, @operation=start, @transition-key=170:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;170:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=117, @rc-code=0, @last-run=1455734926, @last-rc-change=1455734926, @exec-time=2 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/49, version=0.155.842) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-14877-14) Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-14877-14) state:2 Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-14877-14-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-14877-14-header Feb 17 18:48:48 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-14877-14-header Feb 17 18:48:48 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.842 to 0.155.841 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:48 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header lrmd: 2016/02/17_18:48:49 INFO: p_rabbitmq-server: start_rmq_server_app(): blocked access to RMQ port lrmd: 2016/02/17_18:48:49 INFO: get_status() failed with code 2. Command output: Error: unable to connect to node 'rabbit@messaging-node-3': nodedown DIAGNOSTICS =========== attempted to contact: ['rabbit@messaging-node-3'] rabbit@messaging-node-3: * connected to epmd (port 4369) on messaging-node-3 * epmd reports: node 'rabbit' not running at all other nodes on messaging-node-3: ['rabbitmq-cli-14952'] * suggestion: start the node current node details: - node name: 'rabbitmq-cli-14952@node-3' - home dir: /var/lib/rabbitmq - cookie hash: soeIWU2jk2YNseTyDSlsEA== lrmd: 2016/02/17_18:48:49 INFO: p_rabbitmq-server: start_rmq_server_app(): RMQ-runtime (beam) not started, starting... lrmd: 2016/02/17_18:48:50 INFO: p_rabbitmq-server: start_rmq_server_app(): RMQ-server app not started, starting... Feb 17 18:48:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:48:50 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:48:50 INFO: net.ipv4.ip_forward = 1 Feb 17 18:48:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14432) = 0: Resource temporarily unavailable (11) Feb 17 18:48:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:15230 - exited with rc=0 Feb 17 18:48:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:15230:stderr [ -- empty -- ] Feb 17 18:48:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:15230:stdout [ -- empty -- ] Feb 17 18:48:50 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:23 pid:15230 exit-code:0 exec-time:0ms queue-time:0ms lrmd: 2016/02/17_18:48:50 INFO: p_rabbitmq-server: try_to_start_rmq_app(): begin. lrmd: 2016/02/17_18:48:50 INFO: p_rabbitmq-server: try_to_start_rmq_app(): Execute start_app with timeout: 90 lrmd: 2016/02/17_18:48:50 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl start_app >>/var/log/rabbitmq/startup_log 2>&1 Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:53 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:53 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:53 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:53 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:53 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:53 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:53 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:53 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:53 [6775] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: aa511067b35e82b1d836908daecd6c04 for 0.155.842 (0x12f9aa0 0) Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:53 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:53 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:53 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:53 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:53 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:53 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:53 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:53 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:48:59 [15438] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=15438 id=e83efb0f-afa1-463f-bc29-5e5d7d4bd7e0 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-15438-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15438] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15438] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15438] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15438] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:48:59 [15438] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:48:59 [15438] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-15438-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-15438-11) state:2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-15438-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-15438-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-15438-11-header Feb 17 18:48:59 [15441] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=15441 id=1383cc34-0874-4e6c-9df8-1d64de8599c6 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-15441-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15441] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15441] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15441] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15441] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:48:59 [15441] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:48:59 [15441] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-15441-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-15441-11) state:2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-15441-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-15441-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-15441-11-header Feb 17 18:48:59 [15455] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=15455 id=b53a954e-a645-44ba-98de-37bdbe52b50a Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-15455-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15455] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15455] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15455] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15455] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3100 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3100, Current: 3100, Stored: 3100 Feb 17 18:48:59 [15455] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3100 for localhost Feb 17 18:48:59 [15455] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-15455-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-15455-11) state:2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-15455-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-15455-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-15455-11-header Feb 17 18:48:59 [15462] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=15462 id=b5d894b6-f25c-4480-9df8-ad07e8f6f3c6 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-15462-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15462] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15462] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15462] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15462] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:48:59 [15462] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:48:59 [15462] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-15462-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-15462-11) state:2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-15462-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-15462-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-15462-11-header Feb 17 18:48:59 [15463] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=15463 id=68185b24-feb3-46f7-af0f-d869cf2fc66b Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-15463-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15463] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15463] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15463] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15463] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:48:59 [15463] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:48:59 [15463] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-15463-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-15463-11) state:2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-15463-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-15463-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-15463-11-header Feb 17 18:48:59 [15464] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=15464 id=f51752c3-bdde-4abd-9b69-0424a646b6f1 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-15464-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15464] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15464] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15464] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15464] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:48:59 [15464] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:48:59 [15464] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-15464-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-15464-11) state:2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-15464-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-15464-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-15464-11-header Feb 17 18:48:59 [15469] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=15469 id=462820fc-ce0d-468b-af94-334d1475a54b Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-15469-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15469] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15469] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15469] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15469] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.91, Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.91,, Current: 0.83,, Stored: 0.83, Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.91, Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.91,) Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[10]) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 101: cpu_load=0.91, Feb 17 18:48:59 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/101) Feb 17 18:48:59 [15469] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.91, for localhost Feb 17 18:48:59 [15469] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-15469-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-15469-11) state:2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-15469-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-15469-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-15469-11-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:59 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.842 2 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.843 (null) Feb 17 18:48:59 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=843 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.91, Feb 17 18:48:59 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/101, version=0.155.843) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:59 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.843 to 0.155.842 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 101 for cpu_load=0.91, passed Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:59 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1187, version=0.155.843) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:59 [15474] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=15474 id=3c847994-5d80-4fe0-8c4a-8bccb02ce770 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-15474-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15474] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15474] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15474] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15474] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15474] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:48:59 [15474] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-15474-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-15474-11) state:2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-15474-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-15474-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-15474-11-header Feb 17 18:48:59 [15479] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=15479 id=8b93d512-c998-476b-be6a-75090680ccf7 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-15479-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15479] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15479] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15479] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15479] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=250 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 250, Current: 250, Stored: 250 Feb 17 18:48:59 [15479] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=250 for localhost Feb 17 18:48:59 [15479] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-15479-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-15479-11) state:2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-15479-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-15479-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-15479-11-header Feb 17 18:48:59 [15491] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=15491 id=9ba41673-97e5-45c2-9982-aded0f680fb0 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-15491-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15491] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15491] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15491] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15491] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11689 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11689, Current: 11689, Stored: 11689 Feb 17 18:48:59 [15491] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11689 for localhost Feb 17 18:48:59 [15491] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-15491-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-15491-11) state:2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-15491-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-15491-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-15491-11-header Feb 17 18:48:59 [15503] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=15503 id=39a945a0-2e8f-4701-8ba7-de5fc0da89f2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-15503-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15503] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15503] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15503] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15503] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11689 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11689, Current: 11689, Stored: 11689 Feb 17 18:48:59 [15503] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11689 for localhost Feb 17 18:48:59 [15503] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-15503-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-15503-11) state:2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-15503-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-15503-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-15503-11-header Feb 17 18:48:59 [15515] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=15515 id=37692455-bd1a-4909-8023-a8698549ead2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-15515-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15515] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15515] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15515] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15515] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9049 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9049, Current: 9050, Stored: 9050 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9049 Feb 17 18:48:59 [15515] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9049 for localhost Feb 17 18:48:59 [15515] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9049) Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[14]) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:48:59 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/103) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 103: var_log_free=9049 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-15515-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-15515-11) state:2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-15515-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-15515-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-15515-11-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:59 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.843 2 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.844 (null) Feb 17 18:48:59 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=844 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9049 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/103, version=0.155.844) Feb 17 18:48:59 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.844 to 0.155.843 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 103 for var_log_free=9049 passed Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [15527] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:48:59 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:48:59 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1189, version=0.155.844) Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:48:59 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=15527 id=9823f194-9584-439f-aac1-752a1e7461de Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-15527-11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [15527] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15527] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15527] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15527] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:48:59 [15527] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:48:59 [15527] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14432) = 0: Resource temporarily unavailable (11) Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15430 - exited with rc=0 Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-15527-11) Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15430:stderr [ -- empty -- ] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-15527-11) state:2 Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15430:stdout [ arch: x86_64 ] Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15430:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15430:stdout [ free_swap: 3100 ] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15430:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-15527-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-15527-11-header Feb 17 18:48:59 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-15527-11-header Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15430:stdout [ cpu_speed: 6999.99 ] Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15430:stdout [ cpu_cores: 2 ] Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15430:stdout [ cpu_load: 0.91, ] Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15430:stdout [ ram_total: 3050 ] Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15430:stdout [ ram_free: 250 ] Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15430:stdout [ root_free: 11689 ] Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15430:stdout [ root_free: 11689 ] Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15430:stdout [ var_log_free: 9049 ] Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:15430:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:48:59 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:105 pid:15430 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:03 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.844 2 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.845 (null) Feb 17 18:49:03 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=845 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.50 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1191, version=0.155.845) Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[10]) Feb 17 18:49:03 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.845 to 0.155.844 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:49:03 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/105) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:03 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/105, version=0.155.845) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 105 for cpu_load=0.91, passed Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.845 2 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.846 (null) Feb 17 18:49:03 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=846 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9049 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:49:03 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.846 to 0.155.845 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:49:03 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1193, version=0.155.846) Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[14]) Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:49:03 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/107) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:03 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:03 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/107, version=0.155.846) Feb 17 18:49:03 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 107 for var_log_free=9049 passed Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:03 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header lrmd: 2016/02/17_18:49:03 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 2: /usr/sbin/rabbitmqctl start_app >>/var/log/rabbitmq/startup_log 2>&1 lrmd: 2016/02/17_18:49:03 INFO: p_rabbitmq-server: try_to_start_rmq_app(): start_app failed. lrmd: 2016/02/17_18:49:03 WARNING: p_rabbitmq-server: start_rmq_server_app(): RMQ-server app can't start without Mnesia cleaning. Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_dns_monitor_20000 ocf-ns_dns: 2016/02/17_18:49:04 INFO: dnsmasq daemon running Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14432) = 0: Resource temporarily unavailable (11) Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:15684 - exited with rc=0 Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:15684:stderr [ -- empty -- ] Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_dns_monitor_20000:15684:stdout [ -- empty -- ] Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:104 pid:15684 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_heat-engine_monitor_20000 ocf-heat-engine: 2016/02/17_18:49:04 DEBUG: OpenStack Orchestration Engine (heat-engine) monitor succeeded Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14432) = 0: Resource temporarily unavailable (11) Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:15802 - exited with rc=0 Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:15802:stderr [ -- empty -- ] Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_heat-engine_monitor_20000:15802:stdout [ -- empty -- ] Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:110 pid:15802 exit-code:0 exec-time:0ms queue-time:0ms lrmd: 2016/02/17_18:49:04 INFO: get_status(): app rabbit was not found in command output: [{sasl,"SASL CXC 138 11","2.6"}, {stdlib,"ERTS CXC 138 10","2.6"}, {kernel,"ERTS CXC 138 10","4.1"}] lrmd: 2016/02/17_18:49:04 INFO: p_rabbitmq-server: reset_mnesia(): Execute reset with timeout: 90 lrmd: 2016/02/17_18:49:04 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl reset Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_haproxy_monitor_30000 ocf-ns_haproxy: 2016/02/17_18:49:04 DEBUG: p_haproxy: check_ns(): recieved netns list: haproxy ocf-ns_haproxy: 2016/02/17_18:49:04 DEBUG: p_haproxy: get_variables(): set up variables and PIDFILE name ocf-ns_haproxy: 2016/02/17_18:49:04 INFO: haproxy daemon running Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14432) = 0: Resource temporarily unavailable (11) Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:15898 - exited with rc=0 Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:15898:stderr [ -- empty -- ] Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_haproxy_monitor_30000:15898:stdout [ -- empty -- ] Feb 17 18:49:04 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:100 pid:15898 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:49:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14432) = 0: Resource temporarily unavailable (11) Feb 17 18:49:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:15967 - exited with rc=0 Feb 17 18:49:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:15967:stderr [ -- empty -- ] Feb 17 18:49:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:15967:stdout [ -- empty -- ] Feb 17 18:49:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:111 pid:15967 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:05 [6818] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:49:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14432) = 0: Resource temporarily unavailable (11) Feb 17 18:49:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:15992 - exited with rc=0 Feb 17 18:49:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:15992:stderr [ -- empty -- ] Feb 17 18:49:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:15992:stdout [ -- empty -- ] Feb 17 18:49:06 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:112 pid:15992 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:08 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:08 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:08 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:08 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:08 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:08 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:08 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:08 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:08 [6775] node-3.test.domain.local cib: info: cib_process_ping: Reporting our current digest to node-2.test.domain.local: a5c5f8c83efa2191eb0d405227a84951 for 0.155.846 (0x12f9aa0 0) Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:08 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:08 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:08 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:08 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:08 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:08 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:08 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:08 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:13 [6905] node-3.test.domain.local crmd: debug: throttle_cib_load: cib load: 0.003000 (9 ticks in 30s) Feb 17 18:49:13 [6905] node-3.test.domain.local crmd: debug: throttle_load_avg: Current load is 1.610000 (full: 1.61 0.88 0.34 2/542 16294) Feb 17 18:49:13 [6905] node-3.test.domain.local crmd: debug: throttle_io_load: Current IO load is 0.000000 Feb 17 18:49:14 [6818] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:49:14 [16307] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=16307 id=6e48753a-bdb5-4299-a87d-f0ed40cd7a8c Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-16307-11) Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16307] Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:14 [16307] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:14 [16307] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:14 [16307] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:49:14 [16307] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:49:14 [16307] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-16307-11) Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-16307-11) state:2 Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-16307-11-header Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-16307-11-header Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-16307-11-header Feb 17 18:49:14 [16310] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=16310 id=bad56e17-6580-460d-b281-5ece3783abe3 Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-16310-11) Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16310] Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:14 [16310] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:14 [16310] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:14 [16310] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:49:14 [16310] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:49:14 [16310] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-16310-11) Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-16310-11) state:2 Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-16310-11-header Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-16310-11-header Feb 17 18:49:14 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-16310-11-header Feb 17 18:49:15 [16324] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=16324 id=06acd31e-0d1a-49f8-b2db-0ced36074d9e Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-16324-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16324] Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16324] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16324] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16324] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3100 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3100, Current: 3100, Stored: 3100 Feb 17 18:49:15 [16324] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3100 for localhost Feb 17 18:49:15 [16324] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-16324-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-16324-11) state:2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-16324-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-16324-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-16324-11-header Feb 17 18:49:15 [16331] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=16331 id=c1f1db01-fd15-415a-8aaf-32cb3266fa5b Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-16331-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16331] Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16331] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16331] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16331] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:49:15 [16331] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:49:15 [16331] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-16331-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-16331-11) state:2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-16331-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-16331-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-16331-11-header Feb 17 18:49:15 [16332] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=16332 id=ae7b59e0-8571-4c1c-b89c-7856bac9eb3b Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-16332-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16332] Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16332] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16332] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16332] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:49:15 [16332] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:49:15 [16332] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-16332-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-16332-11) state:2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-16332-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-16332-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-16332-11-header Feb 17 18:49:15 [16333] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=16333 id=9e4fd8d9-cd80-4601-bd96-679a19ff8a89 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-16333-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16333] Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16333] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16333] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16333] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:49:15 [16333] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:49:15 [16333] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-16333-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-16333-11) state:2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-16333-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-16333-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-16333-11-header Feb 17 18:49:15 [16338] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=16338 id=cea8febd-67e9-453c-ad54-b83bf13e1dda Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-16338-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16338] Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16338] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16338] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16338] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.88, Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.88,, Current: 0.91,, Stored: 0.91, Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.88, Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.88,) Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[10]) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 109: cpu_load=0.88, Feb 17 18:49:15 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/109) Feb 17 18:49:15 [16338] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.88, for localhost Feb 17 18:49:15 [16338] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [16343] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:15 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.846 2 Feb 17 18:49:15 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.847 (null) Feb 17 18:49:15 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=847 Feb 17 18:49:15 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.88, Feb 17 18:49:15 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/109, version=0.155.847) Feb 17 18:49:15 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.847 to 0.155.846 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 109 for cpu_load=0.88, passed Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x9716f0 for uid=0 gid=0 pid=16343 id=d39aebe2-d02b-4bab-9815-1d5a7e6cce8e Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-16343-12) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16343] Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-16338-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-16338-11) state:2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-16338-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-16338-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-16338-11-header Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [16343] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16343] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16343] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:49:15 [16343] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:49:15 [16343] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-16343-12) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-16343-12) state:2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-16343-12-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-16343-12-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-16343-12-header Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:15 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:15 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1195, version=0.155.847) Feb 17 18:49:15 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:15 [16348] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=16348 id=d34f5034-8570-4fb5-9882-fbc83d358320 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-16348-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16348] Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16348] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16348] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16348] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=250 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 250, Current: 250, Stored: 250 Feb 17 18:49:15 [16348] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=250 for localhost Feb 17 18:49:15 [16348] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-16348-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-16348-11) state:2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-16348-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-16348-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-16348-11-header Feb 17 18:49:15 [16360] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=16360 id=38f0ee02-628c-45a6-b1f3-8054aa9c0681 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-16360-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16360] Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16360] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16360] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16360] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11689 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11689, Current: 11689, Stored: 11689 Feb 17 18:49:15 [16360] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11689 for localhost Feb 17 18:49:15 [16360] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-16360-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-16360-11) state:2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-16360-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-16360-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-16360-11-header Feb 17 18:49:15 [16372] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=16372 id=0a37c996-de78-45ec-bbf8-8bb47eb9bdd6 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-16372-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16372] Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16372] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16372] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16372] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11689 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11689, Current: 11689, Stored: 11689 Feb 17 18:49:15 [16372] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11689 for localhost Feb 17 18:49:15 [16372] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-16372-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-16372-11) state:2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-16372-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-16372-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-16372-11-header Feb 17 18:49:15 [16384] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=16384 id=0a2a98e6-d238-4695-9855-daf2b195ad6e Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-16384-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16384] Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16384] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16384] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16384] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9049 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9049, Current: 9049, Stored: 9049 Feb 17 18:49:15 [16384] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9049 for localhost Feb 17 18:49:15 [16384] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-16384-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-16384-11) state:2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-16384-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-16384-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-16384-11-header Feb 17 18:49:15 [16396] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=16396 id=51e17f49-088d-4b61-b364-2c6789deca47 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-16396-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [16396] Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16396] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16396] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [16396] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:49:15 [16396] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:49:15 [16396] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-16396-11) Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-16396-11) state:2 Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-16396-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-16396-11-header Feb 17 18:49:15 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-16396-11-header Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(14432) = 0: Resource temporarily unavailable (11) Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16299 - exited with rc=0 Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16299:stderr [ -- empty -- ] Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16299:stdout [ arch: x86_64 ] Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16299:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16299:stdout [ free_swap: 3100 ] Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16299:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16299:stdout [ cpu_speed: 6999.99 ] Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16299:stdout [ cpu_cores: 2 ] Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16299:stdout [ cpu_load: 0.88, ] Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16299:stdout [ ram_total: 3050 ] Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16299:stdout [ ram_free: 250 ] Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16299:stdout [ root_free: 11689 ] Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16299:stdout [ root_free: 11689 ] Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16299:stdout [ var_log_free: 9049 ] Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:16299:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:49:15 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:105 pid:16299 exit-code:0 exec-time:0ms queue-time:0ms lrmd: 2016/02/17_18:49:15 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 2: /usr/sbin/rabbitmqctl reset lrmd: 2016/02/17_18:49:15 INFO: p_rabbitmq-server: reset_mnesia(): Execute force_reset with timeout: 90 lrmd: 2016/02/17_18:49:15 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl force_reset lrmd: 2016/02/17_18:49:15 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl force_reset lrmd: 2016/02/17_18:49:15 INFO: p_rabbitmq-server: try_to_start_rmq_app(): begin. lrmd: 2016/02/17_18:49:15 INFO: p_rabbitmq-server: try_to_start_rmq_app(): Execute start_app with timeout: 90 lrmd: 2016/02/17_18:49:15 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl start_app >>/var/log/rabbitmq/startup_log 2>&1 lrmd: 2016/02/17_18:49:16 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl start_app >>/var/log/rabbitmq/startup_log 2>&1 lrmd: 2016/02/17_18:49:16 INFO: p_rabbitmq-server: try_to_start_rmq_app(): start_app was successful. lrmd: 2016/02/17_18:49:16 INFO: p_rabbitmq-server: try_to_start_rmq_app(): waiting for start to finish with timeout: 90 lrmd: 2016/02/17_18:49:16 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl wait /var/run/rabbitmq/p_pid lrmd: 2016/02/17_18:49:17 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl wait /var/run/rabbitmq/p_pid lrmd: 2016/02/17_18:49:17 INFO: p_rabbitmq-server: try_to_start_rmq_app(): start plugins. lrmd: 2016/02/17_18:49:17 INFO: p_rabbitmq-server: try_to_start_rmq_app(): Starting plugins: [rabbitmq_management,rabbitmq_web_dispatch,webmachine,mochiweb, rabbitmq_management_agent,amqp_client] Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:17 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='pingd'] does not exist Feb 17 18:49:17 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 111: pingd=1000 Feb 17 18:49:17 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/111) Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.847 2 Feb 17 18:49:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.848 (null) Feb 17 18:49:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=848 Feb 17 18:49:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']: Feb 17 18:49:17 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/111, version=0.155.848) Feb 17 18:49:17 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.848 to 0.155.847 Feb 17 18:49:17 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 111 for pingd=1000 passed Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:17 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.848 2 Feb 17 18:49:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.849 (null) Feb 17 18:49:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=849 Feb 17 18:49:17 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Feb 17 18:49:17 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1197, version=0.155.849) Feb 17 18:49:17 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.849 to 0.155.848 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:17 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header lrmd: 2016/02/17_18:49:18 INFO: p_rabbitmq-server: start_rmq_server_app(): Execute stop_app with timeout: 90 lrmd: 2016/02/17_18:49:18 DEBUG: p_rabbitmq-server: su_rabbit_cmd(): invoking a command: /usr/sbin/rabbitmqctl stop_app 2>&1 >> "/var/log/rabbitmq/shutdown_log" Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.849 2 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.850 (null) Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=850 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-free_swap']: @value=2650 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1199, version=0.155.850) Feb 17 18:49:18 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.850 to 0.155.849 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='free_swap'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[6]) Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/113) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 lrmd: 2016/02/17_18:49:18 INFO: p_rabbitmq-server: su_rabbit_cmd(): the invoked command exited 0: /usr/sbin/rabbitmqctl stop_app 2>&1 >> "/var/log/rabbitmq/shutdown_log" Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/113, version=0.155.850) Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 113 for free_swap=3100 passed Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[10]) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.850 2 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.851 (null) Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=851 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-cpu_load']: @value=0.49 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1201, version=0.155.851) Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/115) Feb 17 18:49:18 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.851 to 0.155.850 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/115, version=0.155.851) Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 115 for cpu_load=0.88, passed Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.851 2 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.852 (null) Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=852 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-ram_free']: @value=600 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1203, version=0.155.852) Feb 17 18:49:18 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.852 to 0.155.851 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[12]) Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/117) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/117, version=0.155.852) Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 117 for ram_free=250 passed Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.852 2 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.853 (null) Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=853 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-var_log_free']: @value=9048 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1205, version=0.155.853) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.853 to 0.155.852 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[14]) Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/119) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/119, version=0.155.853) Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 119 for var_log_free=9049 passed Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header lrmd: 2016/02/17_18:49:18 INFO: get_status(): app rabbit was not found in command output: [{xmerl,"XML parser","1.3.8"}, {inets,"INETS CXC 138 49","6.0.1"}, {sasl,"SASL CXC 138 11","2.6"}, {stdlib,"ERTS CXC 138 10","2.6"}, {kernel,"ERTS CXC 138 10","4.1"}] lrmd: 2016/02/17_18:49:18 INFO: p_rabbitmq-server: start_rmq_server_app(): RMQ-server app stopped succesfully. lrmd: 2016/02/17_18:49:18 INFO: p_rabbitmq-server: start_rmq_server_app(): RMQ-server app Mnesia cleaned successfully. lrmd: 2016/02/17_18:49:18 INFO: p_rabbitmq-server: master_score(): Updating master score attribute with 1 Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-17366-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [17366] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700, cpd=0x7f08d2c5ffd4 Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f08d2c5a700 Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5690-17366-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-17366-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-17366-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5690-17366-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5690-17366-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5690-17366-26-header Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5690-17366-26-header Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5690-17366-26-header Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-17366-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [17366] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-17366-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-17366-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-17366-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-17366-26-header Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-17366-26-header Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:49:18 [17366] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-17366-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-17366-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-17366-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: crm_client_new: Connecting 0x11389a0 for uid=0 gid=0 pid=17368 id=0febf053-5caa-435a-a185-2a021bf5e44f Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: handle_new_connection: IPC credentials authenticated (6775-17368-14) Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_ipcs_shm_connect: connecting to client [17368] Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:524301; real_size:528384; rb->word_size:132096 Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: cib_native_signon_raw: Connection to CIB successful Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: get_cluster_node_uuid: Result section Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: info: query_node_uuid: Mapped node-3.test.domain.local to 3 Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=17368 id=fb81c394-02bf-4003-bfbd-cb11008a86fe Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-17368-11) Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [17368] Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from crm_attribute: master-p_rabbitmq-server=1 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1, Current: 0, Stored: 0 Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: attrd_update_delegate: Sent update: master-p_rabbitmq-server=1 for node-3.test.domain.local Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of master-p_rabbitmq-server is 1 Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: info: main: Update master-p_rabbitmq-server=1 sent via attrd Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-request-6775-17368-14-header Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-response-6775-17368-14-header Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cib_rw-event-6775-17368-14-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_ipcs_dispatch_connection_request: HUP conn (6775-17368-14) Feb 17 18:49:18 [17368] node-3.test.domain.local crm_attribute: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-17368-14) state:2 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-17368-14-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-17368-14-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-17368-14-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_rabbitmq-server (1) Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_rabbitmq-server'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[2]) Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/121) Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 121: master-p_rabbitmq-server=1 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-17368-11) Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-17368-11) state:2 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-17368-11-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-17368-11-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-17368-11-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 lrmd: 2016/02/17_18:49:18 INFO: p_rabbitmq-server: start_rmq_server_app(): end. Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.853 2 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.854 (null) Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=854 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-master-p_rabbitmq-server']: @value=1 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/121, version=0.155.854) Feb 17 18:49:18 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.854 to 0.155.853 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 121 for master-p_rabbitmq-server=1 passed Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 lrmd: 2016/02/17_18:49:18 INFO: p_rabbitmq-server: start_rmq_server_app(): unblocked access to RMQ port Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1207, version=0.155.854) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header lrmd: 2016/02/17_18:49:18 INFO: p_rabbitmq-server: start: RMQ prepared for start succesfully. lrmd: 2016/02/17_18:49:18 INFO: p_rabbitmq-server: start: action end. Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_start_0:14432 - exited with rc=0 Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:14432:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:14432:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:14432:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:14432:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:14432:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:14432:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:14432:stderr [ Error: {timeout_waiting_for_tables,[rabbit_user,rabbit_user_permission, ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:14432:stderr [ rabbit_vhost,rabbit_durable_route, ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:14432:stderr [ rabbit_durable_exchange, ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:14432:stderr [ rabbit_runtime_parameters, ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_start_0:14432:stderr [ rabbit_durable_queue]} ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_start_0:14432:stdout [ Resetting node 'rabbit@messaging-node-3' ... ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_start_0:14432:stdout [ Forcefully resetting node 'rabbit@messaging-node-3' ... ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_start_0:14432:stdout [ Waiting for 'rabbit@messaging-node-3' ... ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_start_0:14432:stdout [ pid is 15052 ... ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_start_0:14432:stdout [ false ] Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_rabbitmq-server action:start call_id:116 pid:14432 exit-code:0 exec-time:32824ms queue-time:0ms Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_rabbitmq-server after start op complete (interval=0) Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-17406-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [17406] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700, cpd=0x7f08d2c57d84 Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f08d2c5a700 Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5690-17406-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-17406-26) Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5690-17406-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-17406-26) state:2 Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5690-17406-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5690-17406-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5690-17406-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5690-17406-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-17406-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [17406] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-17406-26-header Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-17406-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-17406-26) Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-17406-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-17406-26) state:2 Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [17406] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-17406-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-17406-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-17406-26-header Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed rabbitmq-server_meta-data_0 process 17390 exited with rc=0 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/50) Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_rabbitmq-server_start_0: ok (node=node-3.test.domain.local, call=116, rc=0, cib-update=50, confirmed=true) Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_rabbitmq-server_start_0:116 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\nError: {timeout_waiting_for_tables,[rabbit_user,rabbit_user_permission,\n Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_rabbitmq-server' with start op Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.854 2 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.855 (null) Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=855 Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_rabbitmq-server']/lrm_rsc_op[@id='p_rabbitmq-server_last_0']: @operation_key=p_rabbitmq-server_start_0, @operation=start, @transition-key=89:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;89:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=116, @rc-code=0, @last-run=1455734926, @last-rc-change=1455734926, @exec-time= Feb 17 18:49:18 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/50, version=0.155.855) Feb 17 18:49:18 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.855 to 0.155.854 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:18 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=221:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_rabbitmq-server_notify_0 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=119, reply=1, notify=0, exit=4202464 Feb 17 18:49:18 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_rabbitmq-server action:notify call_id:119 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: info: get_cluster_type: Verifying cluster type: 'corosync' Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: info: get_cluster_type: Assuming an active 'corosync' cluster Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-17434-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [17434] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700, cpd=0x7f08d2c50374 Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f08d2c5a700 Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5690-17434-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-17434-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-17434-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5690-17434-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5690-17434-26-header Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5690-17434-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5690-17434-26-header Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5690-17434-26-header Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-17434-26) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [17434] Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-17434-26-header Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-17434-26-header Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-17434-26-header Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-17434-26) Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:49:18 [17434] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-17434-26) state:2 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-17434-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-17434-26-header Feb 17 18:49:18 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-17434-26-header lrmd: 2016/02/17_18:49:19 INFO: p_rabbitmq-server: notify: post-start begin. lrmd: 2016/02/17_18:49:19 INFO: p_rabbitmq-server: my_host(): hostlist is: node-3.test.domain.local lrmd: 2016/02/17_18:49:19 DEBUG: p_rabbitmq-server: my_host(): comparing 'messaging-node-3' with 'messaging-node-3' lrmd: 2016/02/17_18:49:19 WARNING: p_rabbitmq-server: notify: there are no nodes to join to reported on post-start. Nothing to do. lrmd: 2016/02/17_18:49:19 INFO: p_rabbitmq-server: notify: post-start end. Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_notify_0:17418 - exited with rc=0 Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:17418:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:17418:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:17418:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:17418:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:17418:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:17418:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_notify_0:17418:stdout [ -- empty -- ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_rabbitmq-server action:notify call_id:119 pid:17418 exit-code:0 exec-time:160ms queue-time:0ms Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_rabbitmq-server after notify op complete (interval=0) Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_rabbitmq-server_notify_0 from 221:132:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455734959-13 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_rabbitmq-server_notify_0: ok (node=node-3.test.domain.local, call=119, rc=0, cib-update=0, confirmed=true) Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_rabbitmq-server_notify_0:119 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n ] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for vip__public_start_0 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=41:133:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=vip__public_start_0 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=120, reply=1, notify=0, exit=4202464 Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:vip__public action:start call_id:120 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=50:133:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_conntrackd_monitor_30000 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=121, reply=1, notify=0, exit=4202464 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_conntrackd action:monitor call_id:121 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=159:133:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-dhcp-agent_monitor_20000 Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=122, reply=1, notify=0, exit=4202464 Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_neutron-dhcp-agent action:monitor call_id:122 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=168:133:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_neutron-metadata-agent_monitor_60000 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=123, reply=1, notify=0, exit=4202464 Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_neutron-metadata-agent action:monitor call_id:123 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header ocf-neutron-dhcp-agent: 2016/02/17_18:49:19 DEBUG: OpenStack DHCP Agent (neutron-dhcp-agent) monitor succeeded Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17528) = 0: Resource temporarily unavailable (11) Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17531) = 0: Resource temporarily unavailable (11) Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:17541 - exited with rc=0 Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:17541:stderr [ -- empty -- ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-dhcp-agent_monitor_20000:17541:stdout [ -- empty -- ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:122 pid:17541 exit-code:0 exec-time:39ms queue-time:0ms Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17562) = 0: Resource temporarily unavailable (11) Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_neutron-dhcp-agent after monitor op complete (interval=20000) Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_neutron-dhcp-agent_monitor_20000: ok (node=node-3.test.domain.local, call=122, rc=0, cib-update=51, confirmed=false) Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_neutron-dhcp-agent' with monitor op Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/51) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.855 2 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.856 (null) Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=856 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_neutron-dhcp-agent']: word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.856 2 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.857 (null) Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=857 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_neutron-metadata-agent']: word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700, cpd=0x7f08d2c552f4 Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CPG ] message_handler_req_lib_cpg_finalize cpg finalize for conn=0x7f08d2c5a700 Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-request-5690-17703-26-header Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-response-5690-17703-26-header Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cpg-event-5690-17703-26-header Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: debug: get_local_nodeid: Local nodeid is 3 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-17703-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-17703-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5690-17703-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5690-17703-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5690-17703-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-17703-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [17703] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-17703-26-header Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-17703-26-header Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-17703-26-header Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: info: corosync_node_name: Unable to get node name for nodeid 3 Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: notice: get_node_name: Defaulting to uname -n for the local corosync node name Feb 17 18:49:19 [17703] node-3.test.domain.local crm_node: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-17703-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-17703-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-17703-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-17703-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-17703-26-header Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17528) = 0: Resource temporarily unavailable (11) Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_30000:17531 - exited with rc=0 Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_30000:17531:stderr [ -- empty -- ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_30000:17531:stdout [ vrouter ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_conntrackd_monitor_30000:17531:stdout [ mgmt-conntrd ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:121 pid:17531 exit-code:0 exec-time:161ms queue-time:0ms Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17607) = 0: Resource temporarily unavailable (11) Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17611) = 0: Resource temporarily unavailable (11) Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_conntrackd after monitor op complete (interval=30000) Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_conntrackd_monitor_30000: ok (node=node-3.test.domain.local, call=121, rc=0, cib-update=53, confirmed=false) Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_conntrackd_monitor_30000:121 [ vrouter\n mgmt-conntrd\n ] Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_conntrackd' with monitor op Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/53) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.858 2 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.859 (null) Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=859 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_conntrackd']: word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: do_lrm_rsc_op: Stopped 0 recurring operations in preparation for p_ntp_start_0 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=189:133:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_ntp_start_0 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=126, reply=1, notify=0, exit=4202464 Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: info: log_execute: executing - rsc:p_ntp action:start call_id:126 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:19 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:19 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:19 [6843] node-3.test.domain.local attrd: notice: attrd_cs_dispatch: Update relayed from node-2.test.domain.local Feb 17 18:49:19 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from node-2.test.domain.local: rabbit-master= Feb 17 18:49:19 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header ocf-ns_ntp: 2016/02/17_18:49:19 INFO: ntp daemon is not running lrmd: 2016/02/17_18:49:19 INFO: p_rabbitmq-server: notify: pre-promote begin. ocf-ns_ntp: 2016/02/17_18:49:19 INFO: stdin: is not a tty ocf-ns_ntp: 2016/02/17_18:49:19 INFO: Started ntp daemon. Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17528) = 0: Resource temporarily unavailable (11) Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17607) = 0: Resource temporarily unavailable (11) Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17611) = 0: Resource temporarily unavailable (11) Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_start_0:17779 - exited with rc=0 Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_start_0:17779:stderr [ -- empty -- ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_start_0:17779:stdout [ -- empty -- ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_ntp action:start call_id:126 pid:17779 exit-code:0 exec-time:132ms queue-time:0ms Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_ntp after start op complete (interval=0) lrmd: 2016/02/17_18:49:19 INFO: p_rabbitmq-server: my_host(): hostlist is: node-2.test.domain.local lrmd: 2016/02/17_18:49:19 DEBUG: p_rabbitmq-server: my_host(): comparing 'messaging-node-3' with 'messaging-node-2' Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17528) = 0: Resource temporarily unavailable (11) Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_notify_0:17607 - exited with rc=0 Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:17607:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:17607:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:17607:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:17607:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:17607:stderr [ /usr/bin/umm: line 40: runlevel: command not found ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: notice: operation_finished: p_rabbitmq-server_notify_0:17607:stderr [ -su: 1: [: zrunlevel: unexpected operator ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_rabbitmq-server_notify_0:17607:stdout [ -- empty -- ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: info: log_finished: finished - rsc:p_rabbitmq-server action:notify call_id:124 pid:17607 exit-code:0 exec-time:271ms queue-time:0ms Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17611) = 0: Resource temporarily unavailable (11) Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: info: services_os_action_execute: Managed ns_ntp_meta-data_0 process 18018 exited with rc=0 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_ntp_start_0: ok (node=node-3.test.domain.local, call=126, rc=0, cib-update=54, confirmed=true) Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_ntp' with start op Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/54) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: create_operation_update: send_direct_ack: Updating resource p_rabbitmq-server after notify op complete (interval=0) Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: send_direct_ack: ACK'ing resource op p_rabbitmq-server_notify_0 from 241:133:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c: lrm_invoke-lrmd-1455734959-14 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_rabbitmq-server_notify_0: ok (node=node-3.test.domain.local, call=124, rc=0, cib-update=0, confirmed=true) Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: process_lrm_event: node-3.test.domain.local-p_rabbitmq-server_notify_0:124 [ /usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n/usr/bin/umm: line 40: runlevel: command not found\n-su: 1: [: zrunlevel: unexpected operator\n ] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-29) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.859 2 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.860 (null) Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=860 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_ntp']/lrm_rsc_op[@id='p_ntp_last_0']: @operation_key=p_ntp_start_0, @operation=start, @transition-key=189:133:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;189:133:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=126, @rc-code=0, @last-run=1455734959, @last-rc-change=1455734959, @exec-time=132 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/54, version=0.155.860) Feb 17 18:49:19 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.860 to 0.155.859 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-29-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-29-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-29-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-29) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-29) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-29-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-29-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-29-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=190:133:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=p_ntp_monitor_20000 Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=127, reply=1, notify=0, exit=4202464 Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:p_ntp action:monitor call_id:127 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header ocf-ns_ntp: 2016/02/17_18:49:19 INFO: ntp daemon running Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17528) = 0: Resource temporarily unavailable (11) Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17611) = 0: Resource temporarily unavailable (11) Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:18117 - exited with rc=0 Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:18117:stderr [ -- empty -- ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_ntp_monitor_20000:18117:stdout [ -- empty -- ] Feb 17 18:49:19 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:127 pid:18117 exit-code:0 exec-time:53ms queue-time:0ms Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource p_ntp after monitor op complete (interval=20000) Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation p_ntp_monitor_20000: ok (node=node-3.test.domain.local, call=127, rc=0, cib-update=55, confirmed=false) Feb 17 18:49:19 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'p_ntp' with monitor op Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/55) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:19 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:19 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.860 2 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.861 (null) Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=861 Feb 17 18:49:19 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='p_ntp']: word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:20 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.861 2 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.862 (null) Feb 17 18:49:20 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=862 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='vip__public']/lrm_rsc_op[@id='vip__public_last_0']: @operation_key=vip__public_start_0, @operation=start, @transition-key=41:133:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @transition-magic=0:0;41:133:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c, @call-id=120, @rc-code=0, @last-run=1455734959, @last-rc-change=1455734959, @exec-time=1343 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/crmd/56, version=0.155.862) Feb 17 18:49:20 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.862 to 0.155.861 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:20 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:20 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:20 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:20 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6905-26) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6905] Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:20 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6905] node-3.test.domain.local crmd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:20 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:20 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:20 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6905-26) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-26) state:2 Feb 17 18:49:20 [6905] node-3.test.domain.local crmd: info: do_lrm_rsc_op: Performing key=42:133:0:fdeb7876-00b5-47c4-9ecc-54b1669dbc0c op=vip__public_monitor_5000 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6905-26-header Feb 17 18:49:20 [6818] node-3.test.domain.local pacemaker_remoted: debug: process_lrmd_message: Processed lrmd_rsc_exec operation from 0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa: rc=128, reply=1, notify=0, exit=4202464 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6905-26-header Feb 17 18:49:20 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_execute: executing - rsc:vip__public action:monitor call_id:128 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6905-26-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:20 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:20 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:20 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:20 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:49:20 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.862 2 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.863 (null) Feb 17 18:49:20 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=863 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-master-p_rabbitmq-server']: @value=1000 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:49:20 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.863 to 0.155.862 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:49:20 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1209, version=0.155.863) Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_rabbitmq-server'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[2]) Feb 17 18:49:20 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:49:20 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/123) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:20 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:20 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/123, version=0.155.863) Feb 17 18:49:20 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 123 for master-p_rabbitmq-server=1 passed Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:20 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:20 [6818] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_vrouter_monitor_30000 ocf-ns_vrouter: 2016/02/17_18:49:20 DEBUG: p_vrouter: check_ns(): recieved netns list: vrouter ocf-ns_vrouter: 2016/02/17_18:49:20 INFO: net.ipv4.ip_forward = 1 Feb 17 18:49:20 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(17611) = 0: Resource temporarily unavailable (11) Feb 17 18:49:20 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(18204) = 0: Resource temporarily unavailable (11) Feb 17 18:49:20 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:18226 - exited with rc=0 Feb 17 18:49:20 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:18226:stderr [ -- empty -- ] Feb 17 18:49:20 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_vrouter_monitor_30000:18226:stdout [ -- empty -- ] Feb 17 18:49:20 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:23 pid:18226 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:21 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.863 2 Feb 17 18:49:21 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.864 (null) Feb 17 18:49:21 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=864 Feb 17 18:49:21 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='ping_vip__public']: Feb 17 18:49:21 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1211, version=0.155.865) Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit-master'] does not exist Feb 17 18:49:21 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.865 to 0.155.864 Feb 17 18:49:21 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for rabbit-master=(null) passed Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:49:21 [18247] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:21 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=18247 id=cf515095-1183-46a8-b42d-65cf098541e0 Feb 17 18:49:21 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-18247-11) Feb 17 18:49:21 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18247] Feb 17 18:49:21 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:21 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:21 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:21 [18247] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:21 [18247] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:21 [18247] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:21 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: pingd=1000 Feb 17 18:49:21 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 1000, Current: 1000, Stored: 1000 Feb 17 18:49:21 [18247] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: pingd=1000 for localhost Feb 17 18:49:21 [18247] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:21 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-18247-11) Feb 17 18:49:21 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-18247-11) state:2 Feb 17 18:49:21 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:21 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-18247-11-header Feb 17 18:49:21 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-18247-11-header Feb 17 18:49:21 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-18247-11-header Feb 17 18:49:21 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:17611 - exited with rc=0 Feb 17 18:49:21 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:17611:stderr [ -- empty -- ] Feb 17 18:49:21 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: ping_vip__public_monitor_20000:17611:stdout [ -- empty -- ] Feb 17 18:49:21 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:ping_vip__public action:monitor call_id:125 pid:17611 exit-code:0 exec-time:2047ms queue-time:0ms Feb 17 18:49:21 [6905] node-3.test.domain.local crmd: debug: create_operation_update: do_update_resource: Updating resource ping_vip__public after monitor op complete (interval=20000) Feb 17 18:49:21 [6905] node-3.test.domain.local crmd: notice: process_lrm_event: Operation ping_vip__public_monitor_20000: ok (node=node-3.test.domain.local, call=125, rc=0, cib-update=57, confirmed=false) Feb 17 18:49:21 [6905] node-3.test.domain.local crmd: debug: update_history_cache: Updating history for 'ping_vip__public' with monitor op Feb 17 18:49:21 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/57) Feb 17 18:49:21 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(18204) = 0: Resource temporarily unavailable (11) Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:21 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:21 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:21 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.865 2 Feb 17 18:49:21 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.866 (null) Feb 17 18:49:21 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=866 Feb 17 18:49:21 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='3']/lrm[@id='3']/lrm_resources/lrm_resource[@id='ping_vip__public']: word_size:263168 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:49:24 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:24 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.867 2 Feb 17 18:49:24 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.868 (null) Feb 17 18:49:24 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=868 Feb 17 18:49:24 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-master-p_rabbitmq-server']: @value=1 Feb 17 18:49:24 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1213, version=0.155.868) Feb 17 18:49:24 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.868 to 0.155.867 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:24 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:24 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:49:24 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:49:24 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='master-p_rabbitmq-server'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[2]) Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:49:24 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:49:24 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/126) Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:24 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:24 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/126, version=0.155.868) Feb 17 18:49:24 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 126 for master-p_rabbitmq-server=1 passed Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:24 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:25 [6818] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:49:25 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:18332 - exited with rc=0 Feb 17 18:49:25 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:18332:stderr [ -- empty -- ] Feb 17 18:49:25 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-plugin-openvswitch-agent_monitor_20000:18332:stdout [ -- empty -- ] Feb 17 18:49:25 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:111 pid:18332 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:26 [6818] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of p_neutron-l3-agent_monitor_20000 Feb 17 18:49:26 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:18357 - exited with rc=0 Feb 17 18:49:26 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:18357:stderr [ -- empty -- ] Feb 17 18:49:26 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: p_neutron-l3-agent_monitor_20000:18357:stdout [ -- empty -- ] Feb 17 18:49:26 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:112 pid:18357 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:28 [6818] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of vip__public_monitor_5000 Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:29 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:29 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:29 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:29 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:29 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:29 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:49:29 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:29 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:29 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:29 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:29 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:29 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:49:29 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:49:29 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:49:29 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:49:29 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.868 2 Feb 17 18:49:29 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.869 (null) Feb 17 18:49:29 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=869 Feb 17 18:49:29 [6775] node-3.test.domain.local cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']: Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:49:29 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1215, version=0.155.869) Feb 17 18:49:29 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:29 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='rabbit-start-time'] does not exist Feb 17 18:49:29 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:49:29 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.869 to 0.155.868 Feb 17 18:49:29 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update -6 for rabbit-start-time=(null) passed Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: recurring_action_timer: Scheduling another invokation of sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:49:30 [18423] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=18423 id=42425fb0-2ee7-42f9-8e97-f191f0ed01e7 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-18423-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18423] Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18423] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18423] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18423] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: arch=x86_64 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: x86_64, Current: x86_64, Stored: x86_64 Feb 17 18:49:30 [18423] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: arch=x86_64 for localhost Feb 17 18:49:30 [18423] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-18423-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-18423-11) state:2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-18423-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-18423-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-18423-11-header Feb 17 18:49:30 [18426] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=18426 id=2edb258c-aa6c-4852-baba-5cda3621ad7d Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-18426-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18426] Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18426] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18426] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18426] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: os=Linux-3.13.0-77-generic Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Linux-3.13.0-77-generic, Current: Linux-3.13.0-77-generic, Stored: Linux-3.13.0-77-generic Feb 17 18:49:30 [18426] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: os=Linux-3.13.0-77-generic for localhost Feb 17 18:49:30 [18426] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-18426-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-18426-11) state:2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-18426-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-18426-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-18426-11-header Feb 17 18:49:30 [18440] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=18440 id=0a98477a-326c-433f-a521-417e549eed3c Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-18440-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18440] Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18440] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18440] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18440] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18440] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: free_swap=3100 for localhost Feb 17 18:49:30 [18440] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: free_swap=3100 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3100, Current: 3100, Stored: 3100 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-18440-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-18440-11) state:2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-18440-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-18440-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-18440-11-header Feb 17 18:49:30 [18448] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=18448 id=eec067bd-04c3-4a12-a278-0565ed06c85a Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-18448-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18448] Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18448] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18448] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18448] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Current: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz, Stored: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz Feb 17 18:49:30 [18448] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_info=Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz for localhost Feb 17 18:49:30 [18448] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-18448-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-18448-11) state:2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-18448-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-18448-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-18448-11-header Feb 17 18:49:30 [18449] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=18449 id=03cac138-4666-416b-99ae-cba5685c49ce Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-18449-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18449] Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18449] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18449] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18449] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18449] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_speed=6999.99 for localhost Feb 17 18:49:30 [18449] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_speed=6999.99 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 6999.99, Current: 6999.99, Stored: 6999.99 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-18449-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-18449-11) state:2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-18449-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-18449-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-18449-11-header Feb 17 18:49:30 [18452] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=18452 id=9dfa9917-30c4-4ecd-8acf-c0c81a84803c Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-18452-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18452] Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18452] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18452] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18452] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_cores=2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 2, Current: 2, Stored: 2 Feb 17 18:49:30 [18452] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_cores=2 for localhost Feb 17 18:49:30 [18452] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-18452-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-18452-11) state:2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-18452-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-18452-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-18452-11-header Feb 17 18:49:30 [18457] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=18457 id=2ff14e72-6d1f-4a0a-b98e-2ad93cfbf77b Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-18457-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18457] Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18457] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18457] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18457] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: cpu_load=0.85, Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 0.85,, Current: 0.88,, Stored: 0.88, Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of cpu_load is 0.85, Feb 17 18:49:30 [18457] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: cpu_load=0.85, for localhost Feb 17 18:49:30 [18457] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: cpu_load (0.85,) Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='cpu_load'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[10]) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/129) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.869 2 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.870 (null) Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=870 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-cpu_load']: @value=0.85, Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/129, version=0.155.870) Feb 17 18:49:30 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.870 to 0.155.869 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 129: cpu_load=0.85, Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 129 for cpu_load=0.85, passed Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-18457-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-18457-11) state:2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-18457-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-18457-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-18457-11-header Feb 17 18:49:30 [18462] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=18462 id=5db38eaf-1762-490c-a7d5-3de361b33992 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-18462-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18462] Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:30 [18462] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18462] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18462] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_total=3050 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 3050, Current: 3050, Stored: 3050 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:30 [18462] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_total=3050 for localhost Feb 17 18:49:30 [18462] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-18462-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-18462-11) state:2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-18462-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-18462-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-18462-11-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1217, version=0.155.870) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:30 [18467] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=18467 id=6e60f545-67eb-4d91-b107-dfdabcc14ec8 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-18467-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18467] Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18467] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18467] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18467] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: ram_free=200 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 200, Current: 250, Stored: 250 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of ram_free is 200 Feb 17 18:49:30 [18467] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: ram_free=200 for localhost Feb 17 18:49:30 [18467] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: ram_free (200) Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='ram_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[12]) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/131) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 131: ram_free=200 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-18467-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-18467-11) state:2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-18467-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-18467-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-18467-11-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.870 2 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.871 (null) Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=871 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-ram_free']: @value=200 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/131, version=0.155.871) Feb 17 18:49:30 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.871 to 0.155.870 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 131 for ram_free=200 passed Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [18479] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=18479 id=2f3d6768-606a-4ccb-b575-95b98e6593da Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-18479-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18479] Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [18479] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18479] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18479] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11688 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11688, Current: 11689, Stored: 11689 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of root_free is 11688 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: root_free (11688) Feb 17 18:49:30 [18479] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11688 for localhost Feb 17 18:49:30 [18479] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1219, version=0.155.871) Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[13]) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 133: root_free=11688 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-18479-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-18479-11) state:2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-18479-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-18479-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-18479-11-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/133) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [18491] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=18491 id=04ba6c3a-a461-43d6-b21e-5506e1b59f3b Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-18491-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18491] Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18491] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18491] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [18491] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: root_free=11688 Feb 17 18:49:30 [18491] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: root_free=11688 for localhost Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 11688, Current: 11688, Stored: 11689 Feb 17 18:49:30 [18491] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of root_free is 11688 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: root_free (11688) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.871 2 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.872 (null) Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=872 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-root_free']: @value=11688 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/133, version=0.155.872) Feb 17 18:49:30 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.872 to 0.155.871 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1221, version=0.155.872) Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='root_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[13]) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 135: root_free=11688 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 133 for root_free=11688 passed Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-18491-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-18491-11) state:2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-18491-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-18491-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-18491-11-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/135) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-29) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [18503] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=18503 id=76ac4bd8-c170-427d-b68e-ad0cb485972a Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-18503-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18503] Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18503] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18503] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18503] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-26) state:2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_log_free=9048 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 9048, Current: 9049, Stored: 9049 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: New value of var_log_free is 9048 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: var_log_free (9048) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-26-header Feb 17 18:49:30 [18503] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_log_free=9048 for localhost Feb 17 18:49:30 [18503] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/135, version=0.155.872) Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='3']//transient_attributes//nvpair[@name='var_log_free'] (/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[14]) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: find_nvpair_attr_delegate: Match Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: notice: attrd_perform_update: Sent update 137: var_log_free=9048 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 135 for root_free=11688 passed Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-18503-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-18503-11) state:2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-18503-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-18503-11-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/137) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-18503-11-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-29) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-29) state:2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-29-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-29-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-29-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6843-29) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6843] Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c60180 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1223, version=0.155.872) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:30 [18515] node-3.test.domain.local attrd_updater: info: attrd_update_delegate: Connecting to cluster... 5 retries remaining Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5a700 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: --- 0.155.872 2 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: Diff: +++ 0.155.873 (null) Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib: @num_updates=873 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_perform_op: + /cib/status/node_state[@id='3']/transient_attributes[@id='3']/instance_attributes[@id='status-3']/nvpair[@id='status-3-var_log_free']: @value=9048 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-3.test.domain.local/attrd/137, version=0.155.873) Feb 17 18:49:30 [6776] node-3.test.domain.local stonithd: debug: xml_patch_version_check: Can apply patch 0.155.873 to 0.155.872 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5a700 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6843-29) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-29) state:2 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c60180 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_cib_callback: Update 137 for var_log_free=9048 passed Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_new: Connecting 0x94f9f0 for uid=0 gid=0 pid=18515 id=68eac84d-444b-4eea-81f4-9bf5cb7f84fc Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6843-29-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: handle_new_connection: IPC credentials authenticated (6843-18515-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_shm_connect: connecting to client [18515] Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6843-29-header Feb 17 18:49:30 [18515] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6843-29-header Feb 17 18:49:30 [18515] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [18515] node-3.test.domain.local attrd_updater: debug: qb_rb_open_2: shm size:131085; real_size:135168; rb->word_size:33792 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: update message from attrd_updater: var_lib_mysql_free=18424 Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: attrd_local_callback: Supplied: 18424, Current: 18424, Stored: 18424 Feb 17 18:49:30 [18515] node-3.test.domain.local attrd_updater: debug: attrd_update_delegate: Sent update: var_lib_mysql_free=18424 for localhost Feb 17 18:49:30 [18515] node-3.test.domain.local attrd_updater: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] handle_new_connection IPC credentials authenticated (5690-6775-26) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_dispatch_connection_request: HUP conn (6843-18515-11) Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-18515-11) state:2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_shm_connect connecting to client [6775] Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-18515-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-18515-11-header Feb 17 18:49:30 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-18515-11-header Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: child_waitpid: wait(18393) = 0: Resource temporarily unavailable (11) Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18415 - exited with rc=0 Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18415:stderr [ -- empty -- ] Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18415:stdout [ arch: x86_64 ] Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18415:stdout [ os: Linux-3.13.0-77-generic ] Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18415:stdout [ free_swap: 3100 ] Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18415:stdout [ cpu_info: Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz ] Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18415:stdout [ cpu_speed: 6999.99 ] Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18415:stdout [ cpu_cores: 2 ] Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18415:stdout [ cpu_load: 0.85, ] Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18415:stdout [ ram_total: 3050 ] Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18415:stdout [ ram_free: 200 ] Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18415:stdout [ root_free: 11688 ] Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18415:stdout [ root_free: 11688 ] Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18415:stdout [ var_log_free: 9048 ] Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: sysinfo_node-3.test.domain.local_monitor_15000:18415:stdout [ var_lib_mysql_free: 18424 ] Feb 17 18:49:30 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:105 pid:18415 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_open_2 shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_open_2: shm size:1048589; real_size:1052672; rb->word_size:263168 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_created connection created Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_init_fn lib_init_fn: conn=0x7f08d2c5ddc0 Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [6775] node-3.test.domain.local cib: debug: cib_acl_enabled: CIB ACL is disabled Feb 17 18:49:30 [6775] node-3.test.domain.local cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=node-2.test.domain.local/attrd/1225, version=0.155.873) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-6775-26) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-26) state:2 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2c5ddc0 Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-6775-26-header Feb 17 18:49:30 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-6775-26-header ocf-ns_IPaddr2: 2016/02/17_18:49:31 INFO: ARPING 10.109.1.3 from 10.109.1.4 br-ex Unicast reply from 10.109.1.3 [86:DC:76:8C:5E:C5] 1.151ms Unicast reply from 10.109.1.3 [86:DC:76:8C:5E:C5] 0.553ms Unicast reply from 10.109.1.3 [86:DC:76:8C:5E:C5] 0.551ms Sent 3 probes (1 broadcast(s)) Received 3 response(s) Feb 17 18:49:31 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__public_monitor_5000:18393 - exited with rc=0 Feb 17 18:49:31 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__public_monitor_5000:18393:stderr [ -- empty -- ] Feb 17 18:49:31 [6818] node-3.test.domain.local pacemaker_remoted: debug: operation_finished: vip__public_monitor_5000:18393:stdout [ -- empty -- ] Feb 17 18:49:31 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__public action:monitor call_id:128 pid:18393 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:32 [5319] node-3.test.domain.local corosync notice [MAIN ] sig_exit_handler Node was shut down by a signal Feb 17 18:49:32 [5319] node-3.test.domain.local corosync notice [SERV ] service_exit_schedwrk_handler Unloading all Corosync service engines. Feb 17 18:49:32 [5319] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_withdraw withdrawing server sockets Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_unref qb_ipcs_unref() - destroying Feb 17 18:49:32 [5319] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync vote quorum service v1.0 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-5868-18) state:2 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [CMAP ] cmap_lib_exit_fn exit_fn for conn=0x7f08d2947770 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-response-5690-5868-18-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-event-5690-5868-18-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cmap-request-5690-5868-18-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_withdraw withdrawing server sockets Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_unref qb_ipcs_unref() - destroying Feb 17 18:49:32 [5319] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync configuration map access Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-20) state:2 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cfg-response-5690-6666-20-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cfg-event-5690-6666-20-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cfg-request-5690-6666-20-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_withdraw withdrawing server sockets Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_unref qb_ipcs_unref() - destroying Feb 17 18:49:32 [5319] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync configuration service Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_dispatch_connection_request HUP conn (5690-5868-19) Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-5868-19) state:2 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipc_us_ready: poll(fd 6) got POLLHUP Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7f08d2948620 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: error: cfg_connection_destroy: Connection destroyed Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-5690-5868-19-header Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: notice: pcmk_shutdown_worker: Shuting down Pacemaker Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: notice: stop_child: Stopping crmd: Sent -15 to process 6905 Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: info: crm_signal_dispatch: Invoking handler for signal 15: Terminated Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-5690-5868-19-header Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: notice: crm_shutdown: Requesting shutdown, upper limit is 1200000ms Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: debug: crm_timer_start: Started Shutdown Escalation (I_STOP:1200000ms), src=73 Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: debug: s_crmd_fsa: Processing I_SHUTDOWN: [ state=S_NOT_DC cause=C_SHUTDOWN origin=crm_shutdown ] Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: info: do_shutdown_req: Sending shutdown request to node-2.test.domain.local Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-5690-5868-19-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-23) state:2 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: debug: qb_ipc_us_ready: poll(fd 7) got POLLERR Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f08d2c583e0 Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: error: pcmk_cpg_dispatch: Connection to the CPG API failed: Library error (2) Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: error: crmd_cs_destroy: connection terminated Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-attrd-request-6843-6905-9-header Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-attrd-response-6843-6905-9-header Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-attrd-event-6843-6905-9-header Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: debug: stonith_api_signoff: Signing out of the STONITH Service Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-stonith-ng-request-6776-6905-10-header Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-stonith-ng-response-6776-6905-10-header Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: debug: qb_rb_close: Closing ringbuffer: /dev/shm/qb-stonith-ng-event-6776-6905-10-header Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: debug: get_xpath_object: No match for //st_notify_disconnect in /notify Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5690-6905-23-header Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: info: tengine_stonith_connection_destroy: Fencing daemon disconnected Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: notice: crmd_exit: Forcing immediate exit: Link has been severed (67) Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5690-6905-23-header Feb 17 18:49:32 [6905] node-3.test.domain.local crmd: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5690-6905-23-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6843-28) state:2 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f08d2c5aca0 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5690-6843-28-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5690-6843-28-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5690-6843-28-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6775-25) state:2 Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_ipc_us_ready: poll(fd 6) got POLLHUP Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:49:32 [6775] node-3.test.domain.local cib: error: pcmk_cpg_dispatch: Connection to the CPG API failed: Library error (2) Feb 17 18:49:32 [6775] node-3.test.domain.local cib: error: cib_cs_destroy: Corosync connection lost! Exiting. Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: uninitializeCib: Deallocating the CIB. Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6775) = 0: Resource temporarily unavailable (11) Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6776) = 0: Resource temporarily unavailable (11) Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6818) = 0: Resource temporarily unavailable (11) Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6843) = 0: Resource temporarily unavailable (11) Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6878) = 0: Resource temporarily unavailable (11) Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: error: pcmk_child_exit: Child process crmd (6905) exited: Link has been severed (67) Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: uninitializeCib: The CIB has been deallocated. Feb 17 18:49:32 [6775] node-3.test.domain.local cib: info: terminate_cib: cib_cs_destroy: Exiting fast... Feb 17 18:49:32 [6775] node-3.test.domain.local cib: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: qb_ipc_us_ready: poll(fd 6) got POLLHUP Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-6843-13) state:2 Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-6843-13-header Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: error: pcmk_cpg_dispatch: Connection to the CPG API failed: Library error (2) Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: crit: attrd_cs_destroy: Lost connection to Corosync service! Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-6843-13-header Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: notice: main: Exiting... Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: notice: main: Disconnecting client 0x8a83d0, pid=6905... Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6843-6905-9) state:2 Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-6843-13-header Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-response-6843-6905-9-header Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-event-6843-6905-9-header Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-attrd-request-6843-6905-9-header Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: qb_ipc_us_ready: poll(fd 10) got POLLHUP Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-6843-13-data: No such file or directory (2) Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-6843-13-header: No such file or directory (2) Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-6776-12) state:2 Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-6776-12-header Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-6843-13-data: No such file or directory (2) Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-6843-13-header: No such file or directory (2) Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-6776-12-header Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-6776-12-header Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-6843-13-data: No such file or directory (2) Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-6843-13-header: No such file or directory (2) Feb 17 18:49:32 [6775] node-3.test.domain.local cib: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6775-6905-11) state:2 Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_shm-response-6775-6905-11-header Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: error: attrd_cib_connection_destroy: Connection to the CIB terminated... Feb 17 18:49:32 [6843] node-3.test.domain.local attrd: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_shm-event-6775-6905-11-header Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-cib_shm-request-6775-6905-11-header Feb 17 18:49:32 [6775] node-3.test.domain.local cib: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:49:32 [6775] node-3.test.domain.local cib: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:49:32 [6775] node-3.test.domain.local cib: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: qb_ipc_us_ready: poll(fd 8) got POLLHUP Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: qb_ipc_us_ready: poll(fd 8) got POLLHUP Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: error: crm_ipc_read: Connection to cib_rw failed Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: error: mainloop_gio_callback: Connection to cib_rw[0x167cce0] closed (I/O condition=17) Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: qb_ipc_us_ready: poll(fd 8) got POLLHUP Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-6776-12-data: No such file or directory (2) Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-request-6775-6776-12-header: No such file or directory (2) Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-6776-12-data: No such file or directory (2) Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-response-6775-6776-12-header: No such file or directory (2) Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-6776-12-data: No such file or directory (2) Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: qb_rb_force_close: Force free'ing ringbuffer: /dev/shm/qb-cib_rw-event-6775-6776-12-header: No such file or directory (2) Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: notice: cib_connection_destroy: Connection to the CIB terminated. Shutting down. Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: info: stonith_shutdown: Terminating with 1 clients Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: cib_native_signoff: Signing out of the CIB Service Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6776-6905-10) state:2 Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: crm_client_destroy: Destroying 0 events Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-stonith-ng-response-6776-6905-10-header Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-stonith-ng-event-6776-6905-10-header Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-stonith-ng-request-6776-6905-10-header Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: info: main: Done Feb 17 18:49:32 [6776] node-3.test.domain.local stonithd: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f08d2c52020 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5690-6775-25-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5690-6775-25-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5690-6775-25-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6776-24) state:2 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f08d2c55410 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5690-6776-24-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5690-6776-24-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5690-6776-24-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-21) state:2 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [CPG ] cpg_lib_exit_fn exit_fn for conn=0x7f08d2b4e440 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: qb_ipcs_dispatch_connection_request: HUP conn (6818-6905-7) Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: qb_ipcs_disconnect: qb_ipcs_disconnect(6818-6905-7) state:2 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-response-5690-6666-21-header Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_neutron-plugin-openvswitch-agent_monitor_20000 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-plugin-openvswitch-agent action:monitor call_id:111 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-event-5690-6666-21-header Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: warning: qb_ipcs_event_sendv: new_event_notification (6818-6905-7): Bad file descriptor (9) Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa failed Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-cpg-request-5690-6666-21-header Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_vrouter_monitor_30000 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_vrouter action:monitor call_id:23 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:32 [5319] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_withdraw withdrawing server sockets Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa failed Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_heat-engine_monitor_20000 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_unref qb_ipcs_unref() - destroying Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_heat-engine action:monitor call_id:110 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:32 [5319] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync cluster closed process group service v1.01 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa failed Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6905-27) state:2 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation ping_vip__public_monitor_20000 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:ping_vip__public action:monitor call_id:125 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa failed Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7f08d2c58cb0 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_neutron-l3-agent_monitor_20000 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-l3-agent action:monitor call_id:112 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-5690-6905-27-header Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa failed Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_mysql_monitor_60000 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_mysql action:monitor call_id:103 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa failed Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_neutron-metadata-agent_monitor_60000 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-metadata-agent action:monitor call_id:123 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-5690-6905-27-header Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa failed Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_ntp_monitor_20000 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_ntp action:monitor call_id:127 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa failed Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation vip__public_monitor_5000 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:vip__public action:monitor call_id:128 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-5690-6905-27-header Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa failed Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_dns_monitor_20000 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_dns action:monitor call_id:104 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa failed Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_conntrackd_monitor_30000 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_conntrackd action:monitor call_id:121 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa failed Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_haproxy_monitor_30000 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_haproxy action:monitor call_id:100 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa failed Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation p_neutron-dhcp-agent_monitor_20000 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:p_neutron-dhcp-agent action:monitor call_id:122 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa failed Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: info: cancel_recurring_action: Cancelling operation sysinfo_node-3.test.domain.local_monitor_15000 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: log_finished: finished - rsc:sysinfo_node-3.test.domain.local action:monitor call_id:105 exit-code:0 exec-time:0ms queue-time:0ms Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: warning: send_client_notify: Notification of client crmd/0f8f3c18-c6cf-4f03-b867-0842ca7bb0aa failed Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_disconnect qb_ipcs_disconnect(5690-6666-22) state:2 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: crm_client_destroy: Destroying 14 events Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] _del epoll_ctl(del): Bad file descriptor (9) Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-lrmd-response-6818-6905-7-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_closed cs_ipcs_connection_closed() Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QUORUM] quorum_lib_exit_fn lib_exit_fn: conn=0x7f08d2c52750 Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-lrmd-event-6818-6905-7-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_connection_destroyed cs_ipcs_connection_destroyed() Feb 17 18:49:32 [6818] node-3.test.domain.local pacemaker_remoted: debug: qb_rb_close: Free'ing ringbuffer: /dev/shm/qb-lrmd-request-6818-6905-7-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-response-5690-6666-22-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-event-5690-6666-22-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_rb_close Free'ing ringbuffer: /dev/shm/qb-quorum-request-5690-6666-22-header Feb 17 18:49:32 [5319] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_withdraw withdrawing server sockets Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [QB ] qb_ipcs_unref qb_ipcs_unref() - destroying Feb 17 18:49:32 [5319] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync cluster quorum service v0.1 Feb 17 18:49:32 [5319] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync profile loading service Feb 17 18:49:32 [5319] node-3.test.domain.local corosync notice [SERV ] corosync_service_unlink_and_exit_priority Service engine unloaded: corosync watchdog service Feb 17 18:49:32 [5319] node-3.test.domain.local corosync debug [TOTEM ] memb_leave_message_send sending join/leave message Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:32 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s Feb 17 18:49:32 [5319] node-3.test.domain.local corosync notice [MAIN ] _corosync_exit_error Corosync Cluster Engine exiting normally Feb 17 18:49:33 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:33 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s Feb 17 18:49:38 [6666] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:49:38 [6666] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000111110 (was 00000000000000000000000000111310) Feb 17 18:49:38 [6666] node-3.test.domain.local pacemakerd: info: crm_cs_flush: Sent 0 CPG messages (1 remaining, last=7): Library error (2) Feb 17 18:49:38 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:38 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:49:39 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:39 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s Feb 17 18:49:41 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:41 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s Feb 17 18:49:44 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:44 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s Feb 17 18:49:48 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:48 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s Feb 17 18:49:53 [6666] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:49:53 [6666] node-3.test.domain.local pacemakerd: info: crm_signal_dispatch: Invoking handler for signal 15: Terminated Feb 17 18:49:53 [6666] node-3.test.domain.local pacemakerd: error: pcmk_child_exit: Child process cib (6775) exited: Invalid argument (22) Feb 17 18:49:53 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:53 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:49:54 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:54 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s Feb 17 18:49:56 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:56 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s Feb 17 18:49:59 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:49:59 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s Feb 17 18:50:03 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:03 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s Feb 17 18:50:08 [6666] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:50:08 [6666] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000111010 (was 00000000000000000000000000111110) Feb 17 18:50:08 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:08 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:50:09 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:09 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s Feb 17 18:50:11 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:11 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s Feb 17 18:50:14 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:14 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s Feb 17 18:50:18 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:18 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s Feb 17 18:50:23 [6666] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:50:23 [6666] node-3.test.domain.local pacemakerd: info: pcmk_child_exit: Child process stonith-ng (6776) exited: OK (0) Feb 17 18:50:23 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:23 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:50:24 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:24 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s Feb 17 18:50:26 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:26 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s Feb 17 18:50:29 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:29 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s Feb 17 18:50:33 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:33 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s Feb 17 18:50:38 [6666] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:50:38 [6666] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000011010 (was 00000000000000000000000000111010) Feb 17 18:50:38 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:38 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:50:39 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:39 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s Feb 17 18:50:41 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:41 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s Feb 17 18:50:44 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:44 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s Feb 17 18:50:48 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:48 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s Feb 17 18:50:53 [6666] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:50:53 [6666] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6818) = 0: Connection refused (111) Feb 17 18:50:53 [6666] node-3.test.domain.local pacemakerd: error: pcmk_child_exit: Child process attrd (6843) exited: Transport endpoint is not connected (107) Feb 17 18:50:53 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:53 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:50:54 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:54 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s Feb 17 18:50:56 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:56 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s Feb 17 18:50:59 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:50:59 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s Feb 17 18:51:03 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:51:03 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s Feb 17 18:51:08 [6666] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:51:08 [6666] node-3.test.domain.local pacemakerd: debug: update_node_processes: Node node-3.test.domain.local now has process list: 00000000000000000000000000010010 (was 00000000000000000000000000011010) Feb 17 18:51:08 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:51:08 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 1s Feb 17 18:51:09 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:51:09 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 2s Feb 17 18:51:11 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:51:11 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 3s Feb 17 18:51:14 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:51:14 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 4s Feb 17 18:51:18 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipcc_disconnect: qb_ipcc_disconnect() Feb 17 18:51:18 [6666] node-3.test.domain.local pacemakerd: debug: corosync_cmap_has_config: API connection setup failed: CS_ERR_LIBRARY. Retrying in 5s Feb 17 18:51:23 [6666] node-3.test.domain.local pacemakerd: warning: corosync_cmap_has_config: Could not connect to Cluster Configuration Database API: CS_ERR_LIBRARY (rc=2) Feb 17 18:51:23 [6666] node-3.test.domain.local pacemakerd: debug: child_waitpid: wait(6878) = 0: Connection refused (111) Feb 17 18:51:23 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipc_us_ready: poll(fd 11) got POLLHUP Feb 17 18:51:23 [6666] node-3.test.domain.local pacemakerd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:51:23 [6666] node-3.test.domain.local pacemakerd: info: mcp_quorum_destroy: connection closed Feb 17 18:51:23 [6666] node-3.test.domain.local pacemakerd: debug: pcmk_shutdown_worker: crmd confirmed stopped Feb 17 18:51:23 [6666] node-3.test.domain.local pacemakerd: notice: stop_child: Stopping pengine: Sent -15 to process 6878 Feb 17 18:51:23 [6666] node-3.test.domain.local pacemakerd: debug: qb_ipc_us_ready: poll(fd 9) got POLLHUP Feb 17 18:51:23 [6666] node-3.test.domain.local pacemakerd: debug: _check_connection_state_with: interpreting result -107 (from socket) as a disconnect: Transport endpoint is not connected (107) Feb 17 18:51:23 [6666] node-3.test.domain.local pacemakerd: error: pcmk_cpg_dispatch: Connection to the CPG API failed: Library error (2) Feb 17 18:51:23 [6878] node-3.test.domain.local pengine: info: crm_signal_dispatch: Invoking handler for signal 15: Terminated Feb 17 18:51:23 [6878] node-3.test.domain.local pengine: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:51:23 [6666] node-3.test.domain.local pacemakerd: error: mcp_cpg_destroy: Connection destroyed Feb 17 18:51:23 [6666] node-3.test.domain.local pacemakerd: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:51:23 [6878] node-3.test.domain.local pengine: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:51:23 [6878] node-3.test.domain.local pengine: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:51:25 [6818] node-3.test.domain.local pacemaker_remoted: info: crm_signal_dispatch: Invoking handler for signal 15: Terminated Feb 17 18:51:25 [6818] node-3.test.domain.local pacemaker_remoted: info: lrmd_shutdown: Terminating with 0 clients Feb 17 18:51:25 [6818] node-3.test.domain.local pacemaker_remoted: info: qb_ipcs_us_withdraw: withdrawing server sockets Feb 17 18:51:25 [6818] node-3.test.domain.local pacemaker_remoted: debug: qb_ipcs_unref: qb_ipcs_unref() - destroying Feb 17 18:51:25 [6818] node-3.test.domain.local pacemaker_remoted: info: crm_xml_cleanup: Cleaning up memory from libxml2 Feb 17 18:51:57 [5009] node-3.test.domain.local corosync notice [MAIN ] main Corosync Cluster Engine ('2.3.4'): started and ready to provide service. Feb 17 18:51:57 [5009] node-3.test.domain.local corosync info [MAIN ] main Corosync built-in features: dbus testagents rdma watchdog augeas pie relro bindnow Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize Token Timeout (3650 ms) retransmit timeout (357 ms) Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize token hold (275 ms) retransmits before loss (10 retrans) Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize join (60 ms) send_join (0 ms) consensus (3600 ms) merge (200 ms) Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize downcheck (1000 ms) fail to recv const (2500 msgs) Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize seqno unchanged const (30 rotations) Maximum network MTU 1401 Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize window size per rotation (50 messages) maximum messages per rotation (20 messages) Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize missed count const (5 messages) Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize send threads (2 threads) Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token expired timeout (357 ms) Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token problem counter (2000 ms) Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP threshold (10 problem count) Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP multicast threshold (100 problem count) Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP automatic recovery check timeout (1000 ms) Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP mode set to none. Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize heartbeat_failures_allowed (0) Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize max_network_delay (50 ms) Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0 Feb 17 18:51:57 [5009] node-3.test.domain.local corosync notice [TOTEM ] totemnet_instance_initialize Initializing transport (UDP/IP Unicast). Feb 17 18:51:57 [5009] node-3.test.domain.local corosync notice [TOTEM ] init_nss Initializing transmit/receive security (NSS) crypto: none hash: none Feb 17 18:51:57 [5009] node-3.test.domain.local corosync notice [TOTEM ] timer_function_netif_check_timeout The network interface is down. Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [TOTEM ] main_iface_change_fn Created or loaded sequence id 0.127.0.0.1 for this ring. Feb 17 18:51:57 [5009] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration map access [0] Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cmap [0] Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:51:57 [5009] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cmap Feb 17 18:51:57 [5009] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration service [1] Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cfg [1] Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:51:57 [5009] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cfg Feb 17 18:51:57 [5009] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync cluster closed process group service v1.01 [2] Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cpg [2] Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 17 18:51:57 [5009] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cpg Feb 17 18:51:57 [5009] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync profile loading service [4] Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on pload [4] Feb 17 18:51:57 [5009] node-3.test.domain.local corosync warning [WD ] setup_watchdog No Watchdog, try modprobe Feb 17 18:51:57 [5009] node-3.test.domain.local corosync info [WD ] wd_scan_resources no resources configured. Feb 17 18:51:57 [5009] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync watchdog service [7] Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on wd [7] Feb 17 18:51:57 [5009] node-3.test.domain.local corosync notice [QUORUM] quorum_exec_init_fn Using quorum provider corosync_votequorum Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_readconfig Reading configuration (runtime: 0) Feb 17 18:51:57 [5009] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_read_nodelist_configuration No nodelist defined or our node is not in the nodelist Feb 17 18:51:57 [5009] node-3.test.domain.local corosync crit [QUORUM] quorum_exec_init_fn Quorum provider: corosync_votequorum failed to initialize. Feb 17 18:51:57 [5009] node-3.test.domain.local corosync error [SERV ] corosync_service_defaults_link_and_init Service engine 'corosync_quorum' failed to load for reason 'configuration error: nodelist or quorum.expected_votes must be configured!' Feb 17 18:51:57 [5009] node-3.test.domain.local corosync error [MAIN ] _corosync_exit_error Corosync Cluster Engine exiting with status 20 at service.c:356. Feb 18 08:42:15 [5379] node-3.test.domain.local corosync notice [MAIN ] main Corosync Cluster Engine ('2.3.4'): started and ready to provide service. Feb 18 08:42:15 [5379] node-3.test.domain.local corosync info [MAIN ] main Corosync built-in features: dbus testagents rdma watchdog augeas pie relro bindnow Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize Token Timeout (3650 ms) retransmit timeout (357 ms) Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize token hold (275 ms) retransmits before loss (10 retrans) Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize join (60 ms) send_join (0 ms) consensus (3600 ms) merge (200 ms) Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize downcheck (1000 ms) fail to recv const (2500 msgs) Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize seqno unchanged const (30 rotations) Maximum network MTU 1401 Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize window size per rotation (50 messages) maximum messages per rotation (20 messages) Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize missed count const (5 messages) Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize send threads (2 threads) Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token expired timeout (357 ms) Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token problem counter (2000 ms) Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP threshold (10 problem count) Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP multicast threshold (100 problem count) Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP automatic recovery check timeout (1000 ms) Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP mode set to none. Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize heartbeat_failures_allowed (0) Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize max_network_delay (50 ms) Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0 Feb 18 08:42:15 [5379] node-3.test.domain.local corosync notice [TOTEM ] totemnet_instance_initialize Initializing transport (UDP/IP Unicast). Feb 18 08:42:15 [5379] node-3.test.domain.local corosync notice [TOTEM ] init_nss Initializing transmit/receive security (NSS) crypto: none hash: none Feb 18 08:42:15 [5379] node-3.test.domain.local corosync notice [TOTEM ] timer_function_netif_check_timeout The network interface is down. Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [TOTEM ] main_iface_change_fn Created or loaded sequence id 0.127.0.0.1 for this ring. Feb 18 08:42:15 [5379] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration map access [0] Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cmap [0] Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 18 08:42:15 [5379] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cmap Feb 18 08:42:15 [5379] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration service [1] Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cfg [1] Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 18 08:42:15 [5379] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cfg Feb 18 08:42:15 [5379] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync cluster closed process group service v1.01 [2] Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cpg [2] Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 18 08:42:15 [5379] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cpg Feb 18 08:42:15 [5379] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync profile loading service [4] Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on pload [4] Feb 18 08:42:15 [5379] node-3.test.domain.local corosync warning [WD ] setup_watchdog No Watchdog, try modprobe Feb 18 08:42:15 [5379] node-3.test.domain.local corosync info [WD ] wd_scan_resources no resources configured. Feb 18 08:42:15 [5379] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync watchdog service [7] Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on wd [7] Feb 18 08:42:15 [5379] node-3.test.domain.local corosync notice [QUORUM] quorum_exec_init_fn Using quorum provider corosync_votequorum Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_readconfig Reading configuration (runtime: 0) Feb 18 08:42:15 [5379] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_read_nodelist_configuration No nodelist defined or our node is not in the nodelist Feb 18 08:42:15 [5379] node-3.test.domain.local corosync crit [QUORUM] quorum_exec_init_fn Quorum provider: corosync_votequorum failed to initialize. Feb 18 08:42:15 [5379] node-3.test.domain.local corosync error [SERV ] corosync_service_defaults_link_and_init Service engine 'corosync_quorum' failed to load for reason 'configuration error: nodelist or quorum.expected_votes must be configured!' Feb 18 08:42:15 [5379] node-3.test.domain.local corosync error [MAIN ] _corosync_exit_error Corosync Cluster Engine exiting with status 20 at service.c:356. Feb 18 08:42:16 [5705] node-3.test.domain.local corosync notice [MAIN ] main Corosync Cluster Engine ('2.3.4'): started and ready to provide service. Feb 18 08:42:16 [5705] node-3.test.domain.local corosync info [MAIN ] main Corosync built-in features: dbus testagents rdma watchdog augeas pie relro bindnow Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize Token Timeout (3650 ms) retransmit timeout (357 ms) Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize token hold (275 ms) retransmits before loss (10 retrans) Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize join (60 ms) send_join (0 ms) consensus (3600 ms) merge (200 ms) Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize downcheck (1000 ms) fail to recv const (2500 msgs) Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize seqno unchanged const (30 rotations) Maximum network MTU 1401 Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize window size per rotation (50 messages) maximum messages per rotation (20 messages) Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize missed count const (5 messages) Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize send threads (2 threads) Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token expired timeout (357 ms) Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token problem counter (2000 ms) Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP threshold (10 problem count) Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP multicast threshold (100 problem count) Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP automatic recovery check timeout (1000 ms) Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP mode set to none. Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize heartbeat_failures_allowed (0) Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize max_network_delay (50 ms) Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0 Feb 18 08:42:16 [5705] node-3.test.domain.local corosync notice [TOTEM ] totemnet_instance_initialize Initializing transport (UDP/IP Unicast). Feb 18 08:42:16 [5705] node-3.test.domain.local corosync notice [TOTEM ] init_nss Initializing transmit/receive security (NSS) crypto: none hash: none Feb 18 08:42:16 [5705] node-3.test.domain.local corosync notice [TOTEM ] timer_function_netif_check_timeout The network interface is down. Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [TOTEM ] main_iface_change_fn Created or loaded sequence id 0.127.0.0.1 for this ring. Feb 18 08:42:16 [5705] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration map access [0] Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cmap [0] Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 18 08:42:16 [5705] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cmap Feb 18 08:42:16 [5705] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration service [1] Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cfg [1] Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 18 08:42:16 [5705] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cfg Feb 18 08:42:16 [5705] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync cluster closed process group service v1.01 [2] Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cpg [2] Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 18 08:42:16 [5705] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cpg Feb 18 08:42:16 [5705] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync profile loading service [4] Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on pload [4] Feb 18 08:42:16 [5705] node-3.test.domain.local corosync warning [WD ] setup_watchdog No Watchdog, try modprobe Feb 18 08:42:16 [5705] node-3.test.domain.local corosync info [WD ] wd_scan_resources no resources configured. Feb 18 08:42:16 [5705] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync watchdog service [7] Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on wd [7] Feb 18 08:42:16 [5705] node-3.test.domain.local corosync notice [QUORUM] quorum_exec_init_fn Using quorum provider corosync_votequorum Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_readconfig Reading configuration (runtime: 0) Feb 18 08:42:16 [5705] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_read_nodelist_configuration No nodelist defined or our node is not in the nodelist Feb 18 08:42:16 [5705] node-3.test.domain.local corosync crit [QUORUM] quorum_exec_init_fn Quorum provider: corosync_votequorum failed to initialize. Feb 18 08:42:16 [5705] node-3.test.domain.local corosync error [SERV ] corosync_service_defaults_link_and_init Service engine 'corosync_quorum' failed to load for reason 'configuration error: nodelist or quorum.expected_votes must be configured!' Feb 18 08:42:16 [5705] node-3.test.domain.local corosync error [MAIN ] _corosync_exit_error Corosync Cluster Engine exiting with status 20 at service.c:356. Feb 18 09:00:00 [5388] node-3.test.domain.local corosync notice [MAIN ] main Corosync Cluster Engine ('2.3.4'): started and ready to provide service. Feb 18 09:00:00 [5388] node-3.test.domain.local corosync info [MAIN ] main Corosync built-in features: dbus testagents rdma watchdog augeas pie relro bindnow Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize Token Timeout (3650 ms) retransmit timeout (357 ms) Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize token hold (275 ms) retransmits before loss (10 retrans) Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize join (60 ms) send_join (0 ms) consensus (3600 ms) merge (200 ms) Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize downcheck (1000 ms) fail to recv const (2500 msgs) Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize seqno unchanged const (30 rotations) Maximum network MTU 1401 Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize window size per rotation (50 messages) maximum messages per rotation (20 messages) Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize missed count const (5 messages) Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize send threads (2 threads) Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token expired timeout (357 ms) Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token problem counter (2000 ms) Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP threshold (10 problem count) Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP multicast threshold (100 problem count) Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP automatic recovery check timeout (1000 ms) Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP mode set to none. Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize heartbeat_failures_allowed (0) Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize max_network_delay (50 ms) Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0 Feb 18 09:00:00 [5388] node-3.test.domain.local corosync notice [TOTEM ] totemnet_instance_initialize Initializing transport (UDP/IP Unicast). Feb 18 09:00:00 [5388] node-3.test.domain.local corosync notice [TOTEM ] init_nss Initializing transmit/receive security (NSS) crypto: none hash: none Feb 18 09:00:00 [5388] node-3.test.domain.local corosync notice [TOTEM ] timer_function_netif_check_timeout The network interface is down. Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [TOTEM ] main_iface_change_fn Created or loaded sequence id 0.127.0.0.1 for this ring. Feb 18 09:00:00 [5388] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration map access [0] Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cmap [0] Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 18 09:00:00 [5388] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cmap Feb 18 09:00:00 [5388] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration service [1] Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cfg [1] Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 18 09:00:00 [5388] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cfg Feb 18 09:00:00 [5388] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync cluster closed process group service v1.01 [2] Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cpg [2] Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 18 09:00:00 [5388] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cpg Feb 18 09:00:00 [5388] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync profile loading service [4] Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on pload [4] Feb 18 09:00:00 [5388] node-3.test.domain.local corosync warning [WD ] setup_watchdog No Watchdog, try modprobe Feb 18 09:00:00 [5388] node-3.test.domain.local corosync info [WD ] wd_scan_resources no resources configured. Feb 18 09:00:00 [5388] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync watchdog service [7] Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on wd [7] Feb 18 09:00:00 [5388] node-3.test.domain.local corosync notice [QUORUM] quorum_exec_init_fn Using quorum provider corosync_votequorum Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_readconfig Reading configuration (runtime: 0) Feb 18 09:00:00 [5388] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_read_nodelist_configuration No nodelist defined or our node is not in the nodelist Feb 18 09:00:00 [5388] node-3.test.domain.local corosync crit [QUORUM] quorum_exec_init_fn Quorum provider: corosync_votequorum failed to initialize. Feb 18 09:00:00 [5388] node-3.test.domain.local corosync error [SERV ] corosync_service_defaults_link_and_init Service engine 'corosync_quorum' failed to load for reason 'configuration error: nodelist or quorum.expected_votes must be configured!' Feb 18 09:00:00 [5388] node-3.test.domain.local corosync error [MAIN ] _corosync_exit_error Corosync Cluster Engine exiting with status 20 at service.c:356. Feb 18 09:08:38 [4986] node-3.test.domain.local corosync notice [MAIN ] main Corosync Cluster Engine ('2.3.4'): started and ready to provide service. Feb 18 09:08:38 [4986] node-3.test.domain.local corosync info [MAIN ] main Corosync built-in features: dbus testagents rdma watchdog augeas pie relro bindnow Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totempg_waiting_trans_ack_cb waiting_trans_ack changed to 1 Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize Token Timeout (3650 ms) retransmit timeout (357 ms) Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize token hold (275 ms) retransmits before loss (10 retrans) Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize join (60 ms) send_join (0 ms) consensus (3600 ms) merge (200 ms) Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize downcheck (1000 ms) fail to recv const (2500 msgs) Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize seqno unchanged const (30 rotations) Maximum network MTU 1401 Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize window size per rotation (50 messages) maximum messages per rotation (20 messages) Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize missed count const (5 messages) Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize send threads (2 threads) Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token expired timeout (357 ms) Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP token problem counter (2000 ms) Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP threshold (10 problem count) Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP multicast threshold (100 problem count) Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP automatic recovery check timeout (1000 ms) Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize RRP mode set to none. Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize heartbeat_failures_allowed (0) Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize max_network_delay (50 ms) Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] totemsrp_initialize HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0 Feb 18 09:08:38 [4986] node-3.test.domain.local corosync notice [TOTEM ] totemnet_instance_initialize Initializing transport (UDP/IP Unicast). Feb 18 09:08:38 [4986] node-3.test.domain.local corosync notice [TOTEM ] init_nss Initializing transmit/receive security (NSS) crypto: none hash: none Feb 18 09:08:38 [4986] node-3.test.domain.local corosync notice [TOTEM ] timer_function_netif_check_timeout The network interface is down. Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [TOTEM ] main_iface_change_fn Created or loaded sequence id 0.127.0.0.1 for this ring. Feb 18 09:08:38 [4986] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration map access [0] Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cmap [0] Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 18 09:08:38 [4986] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cmap Feb 18 09:08:38 [4986] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync configuration service [1] Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cfg [1] Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 18 09:08:38 [4986] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cfg Feb 18 09:08:38 [4986] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync cluster closed process group service v1.01 [2] Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init Initializing IPC on cpg [2] Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [MAIN ] cs_get_ipc_type No configured qb.ipc_type. Using native ipc Feb 18 09:08:38 [4986] node-3.test.domain.local corosync info [QB ] qb_ipcs_us_publish server name: cpg Feb 18 09:08:38 [4986] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync profile loading service [4] Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on pload [4] Feb 18 09:08:38 [4986] node-3.test.domain.local corosync warning [WD ] setup_watchdog No Watchdog, try modprobe Feb 18 09:08:38 [4986] node-3.test.domain.local corosync info [WD ] wd_scan_resources no resources configured. Feb 18 09:08:38 [4986] node-3.test.domain.local corosync notice [SERV ] corosync_service_link_and_init Service engine loaded: corosync watchdog service [7] Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [MAIN ] cs_ipcs_service_init NOT Initializing IPC on wd [7] Feb 18 09:08:38 [4986] node-3.test.domain.local corosync notice [QUORUM] quorum_exec_init_fn Using quorum provider corosync_votequorum Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_readconfig Reading configuration (runtime: 0) Feb 18 09:08:38 [4986] node-3.test.domain.local corosync debug [VOTEQ ] votequorum_read_nodelist_configuration No nodelist defined or our node is not in the nodelist Feb 18 09:08:38 [4986] node-3.test.domain.local corosync crit [QUORUM] quorum_exec_init_fn Quorum provider: corosync_votequorum failed to initialize. Feb 18 09:08:38 [4986] node-3.test.domain.local corosync error [SERV ] corosync_service_defaults_link_and_init Service engine 'corosync_quorum' failed to load for reason 'configuration error: nodelist or quorum.expected_votes must be configured!' Feb 18 09:08:38 [4986] node-3.test.domain.local corosync error [MAIN ] _corosync_exit_error Corosync Cluster Engine exiting with status 20 at service.c:356.